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MEMORANDUM  FOR  DIRECTOR,  DEFENSE  SECURITY  ASSISTANCE 

AGENCY 

AUDITOR  GENERAL,  DEPARTMENT  OF  THE  ARMY 


SUBJECT:  Audit  Report  on  Army  Procurement  and  Contract  Administration  Practices 
on  Computer  Software  Service  Contracts  (Report  No.  95-184) 


We  are  providing  this  audit  report  for  review  and  comment.  We  performed  the 
audit  in  response  to  a  congressional  request.  We  considered  management  comments  on 
a  draft  of  this  report  in  preparing  the  final  report. 


DoD  Directive  7650.3  requires  that  all  unresolved  issues  be  resolved  promptly. 
The  Defense  Security  Assistance  Agency  did  not  comment  as  requested  on  the  potential 
monetary  benefits.  As  a  result  of  management  comments,  we  revised  Recommen¬ 
dation  l.c.  to  clarify  our  intention.  Therefore,  we  request  that  the  Defense  Security 
Assistance  Agency  provide  additional  comments  on  Recommendation  l.c.  and 
comment  on  the  potential  monetary  benefits.  We  also  revised  Recommendation  3.b.  in 
response  to  Army  comments.  Those  comments  are  responsive,  and  no  further 
comments  are  required.  We  request  that  Defense  Security  Assistance  Agency  provide 
comments  by  June  30,  1995. 


We  appreciate  the  courtesies  extended  to  the  audit  staff.  Questions  on  the  audit 
should  be  directed  to  Mr.  Terry  L.  McKinney,  Audit  Program  Director,  at  (703) 
604-9288  (DSN  664-9288)  or  Mr.  Ronald  W.  Hodges,  Audit  Project  Manager,  at  (703) 
604-9291  (DSN  664-9291).  See  Appendix  I  for  the  report  distribution.  The  audit  team 
members  are  listed  inside  the  back  cover. 


Robert  JrLieberman 
Assistant  Inspector  General 
for  Auditing 
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Report  No.  95-184  May  2,  1995 

(Project  No.  4CF-5004) 

Army  Procurement  and  Contract  Administration  Practices  on 
Computer  Software  Service  Contracts 

Executive  Summary 


Introduction.  This  audit  was  performed  in  response  to  a  congressional  inquiry 
regarding  the  Army's  procurement  of  computer  software  services.  On  behalf  of  a 
constituent,  Congressman  Jim  Talent  requested  the  Inspector  General,  DoD,  to  review 
the  contracting  practices  of  the  U.S.  Army  Security  Assistance  Command  and  the 
Systems  Integration  and  Management  Activity,  both  subordinate  commands  of  the 
Army  Materiel  Command.  The  constituent  alleged  that  fraud,  waste,  and 
mismanagement  occurred  on  contracts  used  to  develop  computer  software  for  the 
Security  Assistance  Automation,  Army  (SA3),  and  that  the  Army  seldom  or  never  used 
the  developed  software. 

Objectives.  The  audit  objectives  were  to  determine  whether  the  Army  followed 
Federal  and  DoD  acquisition  regulations  when  awarding  and  administering  computer 
software  service  contracts  and  to  review  internal  controls  applicable  to  the  award  and 
administration  of  the  contracts.  To  adequately  answer  the  allegations,  we  amended  the 
audit  objectives  to  focus  on  whether  the  Army's  SA3  development  followed  required 
Defense  and  Army  information  system  development  policies  and  procedures. 
Additionally,  we  assessed  internal  controls  as  they  applied  to  the  development  of  SA3. 

Audit  Results.  The  Army  did  not  follow  Defense  and  Army  policies  and  procedures 
on  information  system  development.  The  Army  spent  more  than  $46  million  on  SA3,  a 
system  that  does  not  fully  satisfy  mission  and  user  requirements.  The  U.S.  Army 
Security  Assistance  Command  SA3  project  manager  and  the  Systems  Integration  and 
Management  Activity  management  did  not  establish  a  sound  SA3  project  baseline  or 
prepare  necessary  cost  information  throughout  the  system's  life  cycle.  As  a  result,  the 
Army  continues  to  spend  approximately  $3.7  million  annually  onSA3,  a  system  that 
does  not  meet  user  needs.  See  Part  II  for  details. 

We  determined  that  about  $3.7  million  annually  in  Security  Assistance  Program  funds 
could  be  put  to  better  use  when  the  Defense  Security  Assistance  Agency  requires  the 
Army  to  justify  all  future  funding  requests  for  SA3.  Justifying  funding  requests  will 
ensure  that  SA3  funds  are  spent  only  for  essential  maintenance  of  SA3.  The  Security 
Assistance  Program  funds  could  be  better  used  by  the  Defense  Security  Assistance 
Agency  to  develop  a  standard  security  assistance  information  system  for  the  Military 
Departments.  Appendix  G  summarizes  the  potential  benefits  resulting  from  the  audit. 

The  allegations  concerning  waste,  mismanagement,  and  the  development  of  useless 
software  were  generally  substantiated.  The  allegations  concerning  fraud  were  not 
substantiated.  Appendix  B  discusses  the  results  of  the  audit  concerning  the 
specific  allegations. 

Summary  of  Recommendations.  We  recommend  that  the  Defense  Security 
Assistance  Agency  discontinue  funding  for  development  of  SA3  in  accordance  with 
DoD  guidance  and  withhold  funding  for  system  maintenance  until  the  U.S.  Army 
Security  Assistance  Command  identifies  and  justifies  essential  maintenance  for  the 


system.  The  Defense  Security  Assistance  Agency  should  fund  only  essential 
maintenance  requirements  identified  in  the  evaluation.  We  also  recommend  that  the 
U.S.  Army  Security  Assistance  Command  and  the  Systems  Integration  and 
Management  Activity  improve  internal  controls  over  the  management  of  SA3 . 

Management  Comments.  The  Defense  Security  Assistance  Agency  agreed  to 
discontinue  all  funding  beyond  essential  maintenance  for  SA3.  The  Defense  Security 
Assistance  Agency  did  not  provide  comments  as  requested  on  the  potential  monetary 
benefits.  The  Army  generally  concurred  with  the  recommendations,  but  disagreed  with 
our  finding,  stating  that  SA3  development  was  completed  in  FY  1990  and  that  SA3 
fully  satisfies  mission  and  user  requirements.  See  Part  II  for  a  summary  of 
management  comments  and  Part  IV  for  the  complete  text  of  management  comments. 

Audit  Response.  Based  on  management  comments,  we  revised  a  recommendation  for 
the  Defense  Security  Assistance  Agency  to  perform  site  visits  within  the  Army  to 
determine  what  resources  are  needed  for  Security  Assistance  automation.  We  also 
modified  a  recommendation  to  the  Army  eliminating  reference  to  limitations  on  internal 
projects.  Although  we  consider  the  Army's  comments  on  the  recommendations 
responsive,  we  disagree  with  the  Army's  position  that  SA3  fully  satisfies  mission  and 
user  needs.  We  request  comments  from  the  Defense  Security  Assistance  Agency  on  the 
unresolved  issues,  including  potential  monetary  benefits,  by  June  30,  1995. 
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Introduction 


Background 


In  September  1993,  Congressman  Jim  Talent  requested  the  Inspector  General, 
DoD,  to  review  the  contracting  practices  of  the  U.S.  Army  Security  Assistance 
Command  (US  AS  AC)  and  the  Army  Systems  Integration  and  Management 
Activity  (SIMA)  on  behalf  of  a  constituent.  The  constituent  alleged  that  fraud, 
waste,  and  mismanagement  occurred  on  computer  software  service  contracts 
used  by  the  Army  to  develop  an  automated  system  for  the  Army  Security 
Assistance  Program. 

Defense  Security  Assistance  Program.  The  Defense  Security  Assistance 
Agency  (DSAA)  has  overall  responsibility  for  administering  the  DoD  Security 
Assistance  Program.  Generally,  the  Army,  the  Navy,  and  the  Air  Force 
execute  foreign  military  sales  (FMS)  cases  in  which  foreign  customers  pay  for 
Defense  goods  and  services.  FMS  cases  generally  require  foreign  customers  to 
pay,  in  advance,  amounts  sufficient  to  cover  all  costs  associated  with  the  sales 
agreements.  DSAA  then  uses  the  funds,  which  are  held  in  an  FMS  trust  fund, 
to  reimburse  the  Military  Departments  for  the  cost  of  executing  and 
administering  FMS  cases. 

Army  Security  Assistance  Program.  The  Commanding  General,  Army 
Materiel  Command  (AMC),  assigns  responsibility  for  the  Army  Security 
Assistance  Program  to  the  Commander,  USASAC.  In  1982,  USASAC  initiated 
development  of  an  automated  information  system,  known  as  Security  Assistance 
Automation,  Army  (SA3),  to  perform  FMS  case  development  and  security 
assistance  management  functions  that  were  previously  performed  manually. 
The  development  of  SA3  was  funded  through  the  FMS  trust  fund. 

Automated  Information  System  Life-Cycle  Management.  DoD  and  Army 
regulations  on  information  system  life-cycle  management  provide  guidance  on 
developing  and  enhancing  automated  information  systems,  such  as  SA3. 
DoD  Instruction  7920.2,  "Automated  Information  System  (AIS)  Life-Cycle 
Management  Review  and  Milestone  Approval  Procedures,"  March  7,  1990,  and 
the  subsequent  DoD  Instruction  8120.2,  "Automated  Information  System  (AIS) 
Life-Cycle  Management  (LCM)  Process,  Review,  and  Milestone  Approval 
Procedures,"  January  14,  1993,  require  Defense  organizations  to  follow  a 
structured  process,  called  life-cycle  management,  for  developing  or  enhancing 
automated  information  systems.  Life-cycle  management  is  intended  to  ensure 
that  Defense  management  is  accountable  for  the  success  or  failure  of  its 
information  systems. 

Army  Regulation  25-3,  "Army  Life-Cycle  Management  of  Information 
Systems,"  November  27,  1989,  establishes  guidelines  and  procedures  that  apply 
to  all  Army  information  systems.  Army  guidelines  for  life-cycle  management 
define  development  phases  and  decision  points  at  which  system  progress  should 
be  assessed  and  documented. 
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SA3  Life-Cycle  Management  and  Funding  Responsibilities.  The  DoD  and 
Army  offices  responsible  for  approving  funds  and  for  providing  life-cycle 
management  and  technical  and  functional  support  for  SA3  are  shown  in 
Figure  1 .  Additional  details  on  S A3  and  the  offices  identified  in  the  figure  are 
discussed  in  Appendix  A. 


Oversight  to  SIMA 
and  USASAC 


Supervision 
and  Control 


Approval  Authority 
for  Funding 


Technical 
Support 

/ 

Security 
Assistance 
Functional 

Support  Office  J  Liaison  Between 
the  Users  and  SIMA 


1  Assistant  Secretary  of  the  Army  (Installations,  Logistics,  and  Environment) 

2  Life-cycle  milestone  approval  channels  through  USASAC  and  AMC  to  the  Assistant  Secretary  of  the  Army 


Figure  1.  Overall  Responsibilities  and  Organization  of  SA3  Development 
and  Management 


Objectives 


The  audit  objectives  were  to  determine  whether  the  Army  followed  Federal  and 
DoD  acquisition  regulations  when  awarding  and  administering  computer 
software  service  contracts  and  to  review  internal  controls  applicable  to  the 
award  and  administration  of  those  contracts.  To  adequately  answer  the 
allegations,  we  amended  the  audit  objectives  to  focus  on  whether  the  Army's 
SA3  development  followed  required  Defense  and  Army  system  development 
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policies  and  procedures.  Additionally,  we  assessed  internal  controls  as  they 
applied  to  the  development  of  SA3.  See  Appendix  B  for  audit  results  in 
response  to  the  allegations. 


Scope  and  Methodology 


Audit  Methodology.  To  answer  the  audit  objective  concerning  the  specific 
allegations,  we  reviewed  documentation  in  the  project  files  for  the  four  specific 
software  applications  identified  in  the  allegations.  We  performed  a  comparison 
test  of  the  SIMA-developed  M204  case  print  program  and  the  contractor- 
developed  M204  case  print  program.  We  also  identified  and  reviewed  four 
additional  software  applications  and  the  M204  conversion  of  the  applications. 

To  answer  the  objective  concerning  management  of  SA3,  we  reviewed  specific 
budget  and  financial  information  to  determine  previous  and  current  funding 
levels  for  SA3.  Also,  we  reviewed  all  available  documentation  that  was 
required  to  support  SA3  development.  Specifically: 

o  at  SIMA,  we  reviewed  the  total  in-house  FMS  funding,  valued  at 
$4.4  million,  from  FYs  1990  through  1994.  We  identified  and  reviewed 
15  task  orders,  valued  at  $2.3  million,  that  were  issued  by  the  General  Services 
Administration  (GSA)  since  FY  1990  for  contractor  technical  support  on  SA3. 
Also,  we  evaluated  1 14  S A3  taskings,  completed  between  October  1990  and 
February  1994,  identified  by  SIMA  as  its  total  SA3  effort,  to  determine 
compliance  with  existing  regulations. 

o  at  USASAC,  we  reviewed  budget  documents  and  policies  concerning 
the  development  and  status  of  SA3.  The  documents  covered  the  entire  life  cycle 
of  SA3  from  FYs  1982  through  1994. 

o  at  DSAA,  we  obtained  copies  of  the  Army's  FYs  1990  through  1994 
FMS  budget  requests,  including  the  Army's  justification  for  requirements  that 
were  partially  or  never  funded  by  DSAA.  We  specifically  reviewed  the 
portions  of  Army  FMS  budget  requests  related  to  SA3,  USASAC,  and  SIMA. 

Audit  Locations.  We  obtained  our  audit  information  primarily  by  examining 
records  and  conducting  interviews  at  the  GSA  Regional  Office,  Fort  Worth, 
Texas;  SIMA-West,  St.  Louis,  Missouri,  and  SIMA-East,  Chambersburg, 
Pennsylvania;  USASAC  headquarters,  Alexandria,  Virginia;  and  USASAC  field 
offices,  New  Cumberland,  Pennsylvania,  and  St.  Louis.  In  addition,  we 
examined  documentation  and  interviewed  personnel  in  the  Computer  Data 
Systems,  Incorporated,  regional  office  in  Fort  Worth  and  in  the  field  office  in 
St.  Louis.  See  Appendix  H  for  a  complete  list  of  organizations  visited 
or  contacted. 

Use  of  Technical  Experts.  We  obtained  software  engineering  assistance  from 
the  Audit  Planning  and  Technical  Support  Directorate,  Assistant  Inspector 
General  for  Auditing,  DoD,  to  evaluate  the  technical  and  engineering  issues  of 
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the  software  applications  reviewed.  The  Quantitative  Methods  Division, 
Assistant  Inspector  General  for  Auditing,  DoD,  provided  statistical  support  for 
selecting  test  cases  in  our  comparison  of  the  SIMA-  and  contractor-developed 
M204  case  print  programs.  See  Appendix  C  for  the  results  of  our  comparison. 

Estimating  Procedures.  Appendix  D  shows  the  calculated  funding  levels  for 
SA3  after  FY  1991.  Starting  in  FY  1992,  SA3  was  not  identified  separately, 
but  was  reported  in  the  USASAC  total  budget  request.  We  used  the  USASAC 
total  budget  request  to  estimate  the  cost  avoidance  that  USASAC  may  recognize 
on  future  SA3  development  efforts. 

Audit  Period  and  Standards.  This  economy  and  efficiency  audit  was 
conducted  from  October  1993  through  September  1994  in  accordance  with 
auditing  standards  issued  by  the  Comptroller  General  of  the  United  States  as 
implemented  by  the  Inspector  General,  DoD.  Accordingly,  the  audit  included 
tests  of  internal  controls  as  necessary. 

Use  of  Computer-Processed  Data.  To  develop  cost  data,  we  relied  on  the 
hours  charged  to  development  and  maintenance  projects  in  the  SIMA  automated 
resource  management  system.  We  tested  the  reliability  of  hours  charged 
(computer-processed  data)  by  comparing  hours  worked  by  employees  with  hours 
charged  against  the  project.  We  also  evaluated  the  procedures  used  by  the 
SIMA  resource  management  system  to  charge  and  record  hours  by  project 
number.  We  did  not  identify  any  significant  errors  or  problems  that  would 
cause  us  to  doubt  the  reliability  of  the  hours  charged. 


Internal  Controls 


Internal  Controls  Reviewed.  The  audit  evaluated  the  implementation  of  the 
DoD  Internal  Management  Control  Program  as  it  related  to  internal  controls 
over  the  development  of  SA3.  Specifically,  we  assessed  existing  DSAA  and 
Army  policy  and  procedures  used  to  oversee  and  control  the  development  and 
maintenance  processes  of  SA3. 

Adequacy  of  Internal  Controls.  The  audit  identified  material  internal  control 
weaknesses  as  defined  by  DoD  Directive  5010.38,  "Internal  Management 
Control  Program,"  April  14,  1987.  Internal  controls  were  not  effective  to 
ensure  that  SA3  was  developed  in  accordance  with  existing  DoD  and 
Army  policy. 

Adequacy  of  Management's  Self-Evaluation.  Although  USASAC  and  SIMA 
had  established  an  internal  management  control  program  and  had  performed 
vulnerability  assessment  reviews,  USASAC  and  SIMA  management  failed  to 
identify  the  internal  control  weaknesses  because  the  internal  management  control 
program  did  not  identify  SA3  as  an  assessable  unit. 

We  could  not  determine  monetary  benefits  associated  with  holding  the  Army 
accountable  for  life-cycle  management  and  funds  expended  on  SA3. 
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Implementing  the  report  recommendations,  however,  should  provide  DSAA  and 
USASAC  with  information  necessary  to  make  cost-effective  decisions  regarding 
the  future  of  SA3.  Recommendations  will  also  ensure  that  SIMA  will  perform 
necessary  planning  before  further  development  and  modification  of  SA3. 
Recommendations  l.d.,  2. a.,  2.b.,  and  3.,  if  implemented,  will  assist  in 
correcting  the  material  internal  control  weaknesses.  Potential  benefits  resulting 
from  implementation  of  the  recommendations  are  in  Appendix  G.  Copies  of  the 
report  will  be  provided  to  the  senior  officials  in  charge  of  internal  controls  for 
DSAA  and  the  Department  of  the  Army. 


Prior  Audits  and  Other  Reviews 


Inspector  General,  DoD.  Report  No.  92-077,  "Software  Development  at 
Central  Design  Activities,"  April  17,  1992.  The  audit  addressed  software 
changes  within  DoD.  The  report  states  that  economic  analyses  were  not 
prepared,  costs  were  not  measured  or  tracked,  identified  benefits  were  not 
achieved,  and  the  Defense  Logistics  Agency  did  not  comply  with  the 
DoD  Accounting  Manual.  The  report  recommended  that  a  standard  cost 
accounting  system  be  developed  and  implemented  by  the  DoD  central  design 
activities  and  that  procedures  for  preparing  and  using  economic  analyses, 
recording  labor  hours,  measuring  costs,  and  achieving  identified  benefits  be 
developed  and  implemented. 

The  Under  Secretary  of  Defense  (Comptroller)  concurred  with  the 
recommendation  to  develop  and  implement  a  single  cost  accounting  system  that 
complies  with  the  DoD  Accounting  Manual.  The  Army  and  the  Defense 
Logistics  Agency  agreed  with  all  recommendations.  The  Navy  and  the 
Air  Force  agreed  with  all  recommendations,  except  the  recommendation 
restricting  overtime  to  only  cost-effective  milestones.  The  recommendation  was 
changed  to  include  authorization  of  overtime  for  DoD  Hotline  and 
mission  priorities. 

Army  Audit  Agency.  Report  No.  MW  90-1,  "Audit  of  System  Change 
Requests  U.S.  Army  Materiel  Command  Systems  Integration  and  Management 
Activity  (Provisional),"  October  26,  1989.  The  report  states  that  cost  and 
benefit  analyses  required  for  information  system  development  and  modification 
were  not  adequate  and  that  an  effective  system  was  not  established  to  validate 
actual  benefits.  In  addition,  some  system  change  requests  were  processed  even 
though  estimated  costs  exceeded  expected  benefits. 

The  report  recommended  that  AMC  establish  effective  guidance  and  procedures 
for  estimating  expected  benefits  and  reporting  actual  benefits.  In  addition, 
AMC  should  halt  development  on  information  systems  until  all  regulatory 
guidance  is  followed  and  all  required  documents  are  prepared.  The  report  also 
recommended  that  AMC  issue  policies  and  procedures  to  ensure  that  all  data  are 
properly  recorded  in  the  systems. 
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AMC  agreed  with  the  recommendations  and  stated  that  guidance  and  procedures 
would  be  established  in  Technical  Bulletin  18-100,  "Army  Automation  Life 
Cycle,"  and  Army  Regulation  25-3  and  that  AMC  major  subordinate  commands 
would  be  directed  to  follow  the  established  guidance  and  procedures.  AMC 
also  stated  that  all  documents  would  be  prepared  or  waivers  would  be  processed 
before  any  further  development. 

Report  No.  MW  88-6,  "Audit  of  Contracting  Services  U.S.  Army  Materiel 
Command  Central  System  Design  Activity  [now  SIMA],  St.  Louis,  Missouri," 
June  1,  1988.  The  report  discusses  deficiencies  in  the  following  areas:  controls 
used  to  monitor  contract  services,  controls  over  funds  used  to  acquire  contract 
services,  and  the  decision  to  use  contracted  services  through  GSA. 

The  report  recommended  that  SIMA  develop  procedures  to  identify  fund  control 
responsibilities  of  personnel  involved  in  contracting  with  GSA  to  include 
maintaining  current  fund  balance  records.  The  report  further  recommended  that 
personnel  responsible  for  billing  verification  should  have  access  to  task  orders 
and  completion  reports.  The  report  also  recommended  that  SIMA  justify  the 
need  for  contracted  programming  support,  including  descriptions  of  all  policies 
and  personnel  responsibilities  on  contracting  for  programming  services.  The. 
justification  should  include  cost  comparisons  of  in-house  and  contract 
alternatives. 

SIMA  agreed  with  the  findings  and  recommendations,  stating  that  all 
recommended  procedures  would  be  established  or  reemphasized  in  the  Design 
Activity  Regulation  715-1,  "Resource  Acquisition,"  June  27,  1988,  which 
included  the  recommended  policies,  procedures,  and  personnel  responsibilities. 


Other  Matters  of  Interest 


The  DoD  Corporate  Information  Management  (CIM)  initiative,  which  began  in 
October  1989,  was  intended  to  achieve  substantial  savings,  in  part,  by 
developing  standard,  Defense-wide  automated  information  systems  for  common 
business  or  functional  areas.  In  November  1990,  the  Assistant  Secretary  of 
Defense  (Command,  Control,  Communications,  and  Intelligence)  assumed 
responsibility  for  CIM  and  all  other  information  management  and  technology 
policies.  The  Assistant  Secretary  established  a  new  office,  the  Director, 
Defense  Information,  with  the  Defense-wide  responsibility  for  implementing 
and  overseeing  the  development  of  standard  information  systems  under  the  DoD 
CIM  initiative. 

The  Director,  Defense  Information,  now  the  Deputy  Assistant  Secretary  of 
Defense  (Information  Management),  devised  a  strategy  for  reviewing  systems 
within  the  DoD  CIM  initiative  that  were  under  development  or  being 
modernized.  The  strategy  required  the  Defense  organization  with  functional 
oversight  responsibility  to  perform  a  business  case  analysis  to  streamline 
business  methods  and  processes  for  selected  operations  within  the  function.  For 
example,  analysis  of  logistics  systems,  including  those  used  in  the  Security 
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Assistance  Program,  is  the  responsibility  of  the  Deputy  Under  Secretary  of 
Defense  (Logistics).  To  comply  with  the  requirements  of  the  DoD 
CIM  initiative,  the  Deputy  Under  Secretary  of  Defense  (Logistics)  established 
the  Joint  Logistics  Systems  Center  in  February  1992  to  implement 
improvements  in  the  business  process  for  the  logistics  function. 
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Part  II  -  Finding  and  Recommendations 


Management  of  Security  Assistance 
Automation,  Army 

The  Army  spent  more  than  $46  million  on  SA3,  a  system  that  does  not 
fully  satisfy  mission  and  user  requirements.  This  wasteful  situation 
occurred  because  of  mismanagement  or  inattention  throughout  the 
SA3  community.  Specifically: 

o  the  SA3  Project  Manager  (PM),  USASAC,  failed  to  follow 
established  Army  regulations  and  guidance  in  developing  and 
maintaining  SA3; 

o  SIMA  management  failed  to  provide  adequate  technical 
support  for  SA3; 

o  Army  management  did  not  provide  adequate  oversight  of 
SA3;  and 

o  DSAA  management  did  not  hold  the  Army  accountable  for 
funds  expended  on  SA3. 

As  a  result,  a  sound  SA3  baseline  was  never  established  and  cost 
information  was  not  available  to  justify  further  enhancements  to  the 
system.  Meanwhile,  USASAC  continued  to  spend  approximately 
$3.7  million  annually  to  develop  and  maintain  SA3,  a  system  that  does 
not  fully  meet  basic  mission  or  user  needs. 


Background 


Information  System  Life-Cycle  Management.  Army  Regulation  25-3 
establishes  the  processes  and  procedures  that  apply  to  the  life  cycle  of  all 
Army  information  systems.  The  regulation  prescribes  milestone  guidance  to 
manage  the  design,  development,  acquisition,  deployment,  operation, 
maintenance,  and  termination  of  a  system,  according  to  the  size  and  complexity 
of  the  system.  To  ensure  adequate  oversight,  the  regulation  categorizes  all 
systems  into  six  classes.  The  assigned  class,  which  is  based  primarily  on 
program  cost,  establishes  the  approval  authority,  management  oversight,  and 
documentation  requirements  for  each  system.  A  Class  I  system,  the  highest 
class,  has  a  cost  threshold  expected  to  exceed  $1  billion.  A  Class  VI  system, 
the  lowest  class,  has  a  cost  threshold  under  $2.5  million. 

Specifically,  when  estimated  program  costs  (costs  incurred  from  justification 
through  total  deployment)  for  system  development  and  modernization  exceed 
$2.5  million  (Class  V  and  higher  systems),  the  regulation  requires  establishment 
of  a  baseline  agreement  plan.  The  baseline  agreement  plan  is  a  formal 
agreement  between  the  program  participants  and  executive  management 
outlining  the  program  requirement,  content,  schedule,  and  cost.  The  baseline 
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agreement  plan  is  then  used  to  help  management  control  program  resource 
changes  such  as  cost  growth,  schedule  slippage,  and  requirement  changes. 
Army  Regulation  25-3  further  requires  the  project  manager  to  provide 
immediate  written  notification  to  the  appropriate  approval  authority  if  a  project 
will  likely  exceed  its  baseline  cost  by  15  percent  in  any  given  phase  or  if  a 
major  schedule  change  is  needed. 

SA3  Information  System.  Before  1983,  the  majority  of  the  Army's  Security 
Assistance  Program  case  development,  management,  and  documentation, 
including  crisis  management,  was  performed  manually  or  using  off-line 
commercial  software  packages.  These  management  methods  used  excessive 
time  and  resources  to  perform  duplicate  entry  and  storage  of  similar  and 
redundant  security  assistance  data,  resulting  in  a  lack  of  standardization  between 
AMC  and  its  major  subordinate  commands. 

To  correct  those  issues,  the  Army  initiated  action  to  develop  SA3.  Basically, 
SA3  was  intended  to  improve  and  centralize  the  security  assistance  case 
development  and  management  by  developing  standard  software  applications  and 
integrating  these  standard  software  applications  with  the  existing  Army 
Commodity  Command  Standard  System. 

SA3  Information  System  Management  Responsibility  and  Oversight. 
According  to  Army  Regulation  25-3,  SA3,  with  program  costs  exceeding 
$10  million,  is  a  class  IV  information  system;  therefore,  all  SA3  life-cycle 
milestones  are  subject  to  approval  by  the  Assistant  Secretary  of  the  Army 
(Installations,  Logistics,  and  Financial  Management),  now  the  Assistant 
Secretary  of  the  Army  (Installations,  Logistics,  and  Environment). 

The  Assistant  Secretary  of  the  Army  (Installations,  Logistics,  and  Environment) 
approved  the  SA3  PM  charter  in  1983.  The  charter  assigned  responsibility  for 
total  management  of  SA3  to  the  SA3  PM.  The  charter  further  required  the 
SA3  PM  to  perform  the  assigned  mission  and  responsibilities  in  accordance  with 
established  Army  life-cycle  management  guidance  provided  in  Army 
Regulation  25-3. 

SA3  Technical  Support.  The  SIMA  Materiel  Management  and  Procurement 
Division  is  responsible  for  the  development  of  AMC  standard  information 
systems.  SIMA  provides  the  functional  and  technical  support  necessary  to 
maintain  and  develop  SA3.  Specifically,  SIMA: 

o  provides  user  assistance,  advice,  and  guidance  during  all  phases  of 
life-cycle  development; 

o  develops  various  system  and  life-cycle  management  documentation; 
o  develops  and  monitors  multi-level  application  testing; 
o  develops  statements  of  work;  and 
o  monitors  contractor  performance. 
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Resources  Spent  Without  Satisfying  Mission  or 
User  Requirements 


Resources  Invested  in  SA3.  From  FYs  1983  through  1994,  the  Army  spent 
more  than  $46  million*  on  SA3,  although  the  latest  approved  cost  estimate  from 
September  1982  indicated  that  the  Army  originally  planned  to  invest  only  about 
$18.3  million  in  SA3  over  an  estimated  8-year  system  life  cycle. 

The  Army  has  little  to  show  for  the  $46  million  invested  in  SA3.  Figure  2 
shows  four  of  the  major  SA3  system  applications  that  were  part  of  the  initial 
SA3  plan.  These  initial  major  SA3  system  applications  were: 

o  Case  Development, 

o  Case  Management, 

o  International  Logistics  Supply  Delivery  Plan,  and 
o  AMC  Standard  FMS  Pricing  System. 

Two  additional  major  SA3  applications,  the  Security  Assistance  Management 
Acquisition  Program  and  the  Communication s-Electronics  Command  version  of 
the  International  Logistics  Supply  Delivery  Plan  (under  development),  and 
two  minor  SA3  applications,  760  and  1404,  are  also  shown  in  Figure  2.  See 
page  24  and  Appendix  B  for  details  on  the  two  minor  SA3  applications. 


*The  Army  was  unable  to  provide  documentation  to  support  a  breakdown  of 
SA3  development  and  enhancement  costs  versus  maintenance  costs  through 
FY  1994.  The  SA3  PM  reported  $34.9  million  spent  on  SA3  through  FY  1990, 
the  year  the  Army  reported  SA3  a  fielded  system.  The  $34.9  million  did  not 
include  the  cost  to  develop  the  M204  conversion  ($3.4  million)  or  the  Security 
Assistance  Management  Acquisition  Program  ($651,000).  See  discussion  on 
Reporting  SA3  Information  System  Cost,  page  17. 
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SA3  System 


^  Army  Tank-Automotive  Command 
^Army  Missile  Command 
3  Army  Communications-Electronics  Command 


Applications  not  Integrated  with  SA3 


Figure  2.  SA3  System  Applications  That  Were  Developed 

Of  the  four  major  SA3  systems  applications  that  were  reported  as  completed, 
some  were  seldom  used,  were  developed  without  use  of  SA3  resources,  or  were 
not  fully  integrated  into  SA3.  For  example,  the  International  Logistics  Supply 
Delivery  Plan,  one  of  four  completed  major  SA3  systems  applications,  was 
seldom  used  and  was  deleted  in  1991.  The  application  was  supposed  to  provide 
case  managers  a  system  of  monitoring  case  status  (primarily  tracking  customer 
requisitions)  once  a  case  had  been  implemented. 

Two  additional  attempts  were  made  to  satisfy  the  same  requirement.  The 
Security  Assistance  Management  Acquisition  Program  was  developed  at  the 
Missile  Command  (an  AMC  major  subordinate  command)  at  a  cost  of 
$651,000  over  a  3-year  period.  Phase  I  of  this  application  was  fielded  in 
March  1993  but  experienced  little,  if  any,  use  and  has  not  been  integrated 
with  SA3.  During  June  1994,  we  determined  that  the  Communications- 
Electronics  Command  (an  AMC  major  subordinate  command)  had  tasked  a 
contractor  to  develop  a  similar  application  for  internal  use,  a  third  attempt  to 
satisfy  this  same  functional  need.  The  Communications-Electronics  Command 
application  was  intended  for  use  only  within  the  Communications-Electronics 
Command  and  was  not  scheduled  to  be  integrated  with  SA3.  In  addition, 
SA3  integration  was  not  complete,  as  only  the  Case  Development  and  Case 
Management  SA3  system  applications  were  fully  integrated  into  SA3,  which  is  a 
part  of  the  Army  Commodity  Command  Standard  System. 
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Failure  to  Satisfy  Mission  or  User  Requirements.  SA3  had  limited 
capabilities  and  did  not  fully  satisfy  basic  Security  Assistance  Program  mission 
or  user  requirements.  Review  of  existing  documentation  and  discussions  with 
case  developers  and  case  managers  at  six  (now  five)  AMC  major  subordinate 
commands  showed  that  SA3  did  not  correct  deficiencies  that  existed  under  the 
prior  system.  The  following  deficiencies  are  examples  of  the  SA3  failure  to 
meet  user  needs. 

o  Users  continued  to  develop  a  significant  number  of  cases  off-line, 
without  using  SA3. 

o  Users  managed  cases  primarily  off-line. 

o  Users  duplicated  entry  and  storage  of  similar  and  redundant  data. 

As  a  result  of  the  three  deficiencies,  USASAC  could  not  use  SA3  to  centralize 
or  standardize  management  of  the  Security  Assistance  Program  or  to  avoid 
duplicate  efforts  of  managing  the  Security  Assistance  Program.  In  addition, 
users  at  the  AMC  major  subordinate  commands  continued  to  develop  their  own 
applications  to  satisfy  needs  not  met  by  SA3.  Consequently,  SA3  did  not  fully 
satisfy  basic  mission  requirements. 

Users  Continued  to  Develop  Cases  Off-Line.  In  April  1994,  11  years 
into  the  SA3  life  cycle,  users  were  just  beginning  to  use  SA3  to  develop  cases. 
Users  provided  several  reasons  why  they  previously  elected  not  to  use  SA3  to 
develop  their  cases.  The  most  frequent  comment  made  was  that  SA3  was  fine 
for  simple  cases  but  that  the  system  was  too  rigid  to  accommodate  unique  or 
complex  cases.  In  addition,  users  expressed  frustration  with  having  to  wait 
overnight  for  information  requested  from  the  system.  For  example,  of  the 
average  of  1,100  cases  developed  by  the  Missile  Command  between 
October  1990  and  April  1994,  674  (61  percent)  were  developed  off-line,  using  a 
commercial  software  package. 

In  January  1990,  USASAC,  aware  that  users  were  not  using  SA3,  tasked  SIMA 
to  upgrade  SA3.  The  upgrade,  which  became  known  as  the  M204  conversion, 
primarily  converted  the  existing  SA3  data  base  from  S2K,  a  hierarchical  format, 
to  M204,  a  relational  format.  This  conversion  provided  users  a  quicker  and 
more  user-friendly  method  of  retrieving  data.  However,  the  M204  conversion 
also  affected  all  previously  developed  SA3  applications  and  related  programs 
that  used  the  S2K  format.  For  example,  one  of  the  programs  that  formed  the 
case  development  application  was  the  case  print  program.  This  program 
performed  the  print  function  for  the  case  development  application.  When  SIMA 
was  tasked  to  convert  the  SA3  data  base  from  the  S2K  format  to  M204  format, 
the  case  print  program  had  to  be  redesigned  to  be  compatible  with  the 
M204  format. 

Case  managers  at  US  AS  AC-New  Cumberland,  responsible  for  managing 
30  percent  of  the  Army's  average  6,000  open  FMS  cases,  did  not  use  SA3  to 
develop  cases  before  June  1994.  When  the  SA3  prototype  was  first  tested  at 
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US  AS  AC-New  Cumberland  in  1991,  users  tried  to  develop  and  print  cases 
using  SA3,  but  most  became  frustrated  and  chose  to  use  an  off-line  commercial 
software  package  to  develop  and  print  their  cases. 

Although  SA3  development  began  in  1983,  users  only  recently  began  using  the 
system  on  a  regular  basis  to  develop  and  print  their  cases.  SIMA  fielded  the 
SA3  M204  database  conversion  to  the  five  AMC  major  subordinate  commands 
in  April  1994  and  to  USASAC-New  Cumberland  in  May  1994.  According  to 
users,  the  SA3  M204  database  conversion  made  developing  cases  easier; 
however,  shortfalls  in  SA3  still  prevented  users  from  using  SA3  to  manage 
their  cases. 

Users  Continued  to  Manage  Cases  Off-Line.  Users  at  the  five  AMC 
major  subordinate  commands  managed  their  cases  off-line  because  the  capability 
to  manage  cases  using  SA3  was  never  successfully  developed.  Case 
management  is  the  ability  to  manage  a  case  from  implementation  through  close 
out  by  measuring  the  performance  and  status  of  a  foreign  customer's  goods 
and  services. 

Two  attempts  were  made  to  provide  SA3  with  case  management  capabilities. 

o  The  International  Logistics  Supply  Delivery  Plan  was  the  original 
attempt  to  provide  a  management  application.  This  application,  fielded  in  1989, 
was  designed  to  provide  case  tracking  and  reporting  capabilities.  The 
application  was  canceled  after  a  March  1991  study  determined  that  the 
application  was  not  being  used. 

o  The  Missile  Command  Security  Assistance  Management  Acquisition 
Program  was  the  second  attempt  to  develop  case  management  capabilities  within 
SA3.  Although  fielded,  this  application  received  minimal  use.  As  of 
July  1994,  users  still  had  not  received  training,  and  they  expressed  skepticism  as 
to  whether  this  latest  application  would  provide  the  necessary 
management  capabilities.  Further,  the  Communications-Electronics  Command 
initiated  development  efforts  for  its  own  case- tracking  system,  also  known  as 
the  International  Logistics  Supply  Delivery  Plan. 

Users  Duplicate  Entry  and  Storage  of  Data.  SA3  did  not  eliminate  the 
need  for  duplicate  entry  and  storage  of  similar  and  redundant  data.  The 
recording  of  basic  case  information  and  the  capability  of  the  AMC  Standard 
Foreign  Military  Sales  Pricing  System  provide  significant  examples  of  the 
continued  duplication  of  effort  in  managing  the  Security  Assistance  Program. 

Cases  are  managed  and  tracked  in  varying  levels  of  detail  at  several  AMC  major 
subordinate  commands,  but  the  basic  case  information  is  needed  at  all  AMC 
major  subordinate  commands.  The  basic  information  for  every  implemented 
case  includes  case  and  country  designator;  case  description;  case  value; 
implementing  agency;  and  significant  action  dates  such  as  dates  of  offer, 
acceptance,  and  implementation.  This  basic  information  is  entered  by  users  at 
least  three  times  on  different  information  systems,  as  shown  in  Figure  3. 
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Basic  case  information,  along  with  other  necessary  detailed  information,  is 
entered  first  at  an  AMC  major  subordinate  command,  where  the  case  is 
developed  and  managed  either  on  SA3  or  off-line.  The  case  is  printed  and  sent 
to  US  AS  AC-New  Cumberland,  where  the  same  basic  information  is  entered  on 
the  Centralized  Integrated  System-International  Logistics  to  maintain  overall 
case  tracking  and  management  information  for  the  country  case  manager.  The 
case  is  then  sent  to  USASAC-Alexandria,  where  the  same  basic  case 
information  is  entered  again  on  a  third  system,  the  Security  Assistance  Case 
Tracking  System.  This  tracking  system  maintains  overall  case  tracking  and 
management  information  for  the  country  program  manager.  The  country 
program  manager  oversees  all  requests  and  cases  for  a  specific  country  and 
delegates  the  responsibility  of  overseeing  cases  to  subordinate  country 
case  managers. 


Figure  3.  Basic  Case  Information  Entered  Three  Separate  Places 

Without  a  centralized  security  assistance  data  base  of  FMS  cases,  the  Army  will 
continue  to  waste  limited  resources  through  duplicate  efforts  to  manage  the 
Security  Assistance  Program.  Unsuccessful  attempts  were  made  to  integrate  the 
International  Logistics  Supply  Delivery  Plan.  Integrating  the  two  related 
applications  at  Missile  Command  and  Communications-Electronics  Command 
discussed  earlier  have  not  been  attempted. 

Capability  of  Standard  Pricing  System.  One  intended  goal  of  the  AMC 
Standard  FMS  Pricing  System  was  to  eliminate  the  need  for  dual  entry  when 
pricing  FMS  cases.  The  AMC  Standard  FMS  Pricing  System,  one  of  the  four 
completed  major  SA3  system  applications  (see  Figure  2),  was  developed  at  the 
Tank-Automotive  Command  (an  AMC  major  subordinate  command)  as  a  stand¬ 
alone  system  and,  subsequently,  was  made  available  to  the  other  AMC  major 
subordinate  commands.  Users  at  the  various  AMC  major  subordinate 
commands  considered  the  pricing  feature  of  the  AMC  Standard  FMS  Pricing 
System  to  be  useful;  however,  the  current  version  of  the  pricing  system  has  not 
been  integrated  with  the  new  M204  data  base.  For  example,  although  the  AMC 
Standard  FMS  Pricing  System  was  integrated  into  the  SA3  S2K  data  base,  it 
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was  seldom  used  because  the  process  used  to  retrieve  pricing  information  from 
the  S2K  data  base  took  too  long.  As  a  consequence,  users  had  to  extract  pricing 
data  from  the  standard  pricing  system  and  re-enter  the  applicable  prices  when 
developing  a  case.  Also,  when  SA3  was  converted  to  a  M204  data  base,  the 
Tank-Automotive  Command  was  tasked  by  USASAC  to  develop  another  pricing 
application  to  be  used  with  the  SA3  M204  data  base,  but  the  application  has  not 
been  integrated  with  SA3.  As  a  result,  the  AMC  Standard  FMS  Pricing  System 
failed  to  satisfy  SA3  mission  requirements  and  to  achieve  the  original  goal  of 
eliminating  duplicate  entries  for  pricing  FMS  cases. 


Project  Management  to  Develop  and  Maintain  the  SA3 
Information  System 


The  SA3  PM  did  not  follow  established  Army  life-cycle  management  policies 
and  processes  during  system  development.  In  some  cases,  the  SA3  PM  did  not 
report  required  information  to  Army  and  USASAC  management,  or  information 
reported  to  management  was  inaccurate  or  misleading.  As  a  result, 
management  did  not  have  a  sound  basis  for  evaluating  the  status  of  SA3.  In 
other  instances,  USASAC  failed  to  address  problems,  even  when  available 
information  indicated  significant  system  deficiencies,  cost  overruns,  and 
schedule  slippages.  The  following  examples  summarize  the  inadequacy  of 
information  the  SA3  PM  reported  to  management,  and  management's  failure  to 
respond  to  problem  indicators. 

Reporting  SA3  Information  System  Cost.  The  SA3  PM  did  not  prepare  cost 
estimates  or  report  significant  cost  increases  for  SA3,  as  required  by  Army 
guidance  on  life-cycle  management.  Army  regulations  require  that  the 
approving  official  receive  immediate  written  notification  if  a  project  is  expected 
to  exceed  its  baseline  cost  by  15  percent  in  any  given  phase.  The  mission 
element  needs  statement,  approved  in  September  1982  by  the  Assistant 
Secretary  of  the  Army  (Installations,  Logistics,  and  Environment),  was  the  last 
approved  estimate  of  the  projected  investment  cost  for  SA3. 

The  September  1982  mission  element  needs  statement  showed  estimated 
SA3  software  development  and  procurement  costs  to  be  $18.3  million  through 
full  extension  of  the  SA3  to  all  planned  sites.  The  inaccuracy  of  this 
information  should  have  been  evident  when  the  actual  cost  information  in  the 
USASAC  May  1989  management  plan  was  much  higher.  For  example,  the 
May  1989  management  plan  showed  that,  as  of  May  1989,  year  7  of  an 
estimated  8-year  life  cycle,  the  Army  had  invested  $25.6  million  in  SA3, 
exceeding  the  latest  approved  baseline  estimate  ($18.3  million  in  1982)  by 
40  percent.  The  USASAC  May  1989  management  plan  also  reported  that,  as  of 
May  1989,  the  required  system  programming  was  only  33  percent  complete, 
when,  in  fact,  the  system  programming  should  have  been  at  least  75  percent 
complete.  See  Figure  4. 
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Figure  4.  Actual  SA3  Development  and  Procurement  Costs  Exceeded 
Estimated  Costs 

Information  prepared  16  months  later  indicated  that  the  SA3  PM  continued  to 
spend  money  for  SA3  without  revising  and  reporting  cost  estimates,  while 
receiving  little  in  return.  For  example,  in  an  October  1990  mid-term  analysis 
report  of  SA3,  the  SA3  PM  office  reported  that  $34.9  million  was  spent  on 
SA3,  with  only  6  of  21  planned  system  applications  completed.  The  total  spent 
as  of  October  1990  exceeded  the  baseline  estimate  by  91  percent.  Further,  the 
$34.9  million  did  not  include  one  of  the  four  completed  applications,  the  AMC 
Standard  FMS  Pricing  System.  This  application  was  developed  at  the 
Tank- Automotive  Command  without  the  use  of  SA3  resources,  and  the  cost  to 
develop  the  application  was  unknown. 

Although  the  October  1990  SA3  PM  mid-term  analysis  report,  the  last  formal 
documentation  prepared  by  the  SA3  PM  on  the  status  of  SA3,  showed  that  the 
SA3  project  had  significant  cost  overruns  and  schedule  stoppages,  the  SA3  PM 
never  presented  this  report  to  Army  management.  Without  current  information 
regarding  the  cost  and  status  of  SA3,  Army  management  could  not  determine 
the  most  cost-effective  alternative  for  completing  the  project.  The  SA3  PM 
stated  that  Army  management  generally  considered  security  assistance  a  low 
priority  area;  therefore,  any  briefings  that  were  presented  to  management 
usually  occurred  after  action  was  already  taken. 

Documentation  of  Major  Modification.  The  SA3  PM  did  not  provide 
required  documentation  or  obtain  necessary  approval  in  support  of  the 
SA3  M204  conversion,  a  major  SA3  modification  costing  at  least  $3.4  million 
over  4  years.  The  $3.4  million  estimate  is  a  conservative  rate  based  on  an 
average  SIMA-computed  rate  of  $29  per  hour.  Current  Army  guidance  suggests 
that  a  rate  of  $55  per  hour  would  be  more  accurate. 
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In  January  1990,  USASAC  directed  SIMA  to  convert  the  existing  SA3  data  base 
from  S2K,  a  hierarchical  format,  to  M204,  a  relational  format.  USASAC 
levied  the  tasking  with  little,  if  any,  analysis  on  what  resources  were  required, 
how  long  the  effort  would  take,  how  much  it  would  cost,  and  what  the 
anticipate!  benefits  would  be. 

Failure  to  Evaluate  Feasibility  of  Modification.  The  SA3  PM  did  not 
evaluate  and  report  the  feasibility  of  converting  to  M204  or  the  overall  effect 
the  M204  conversion  had  on  the  existing  SA3  software  applications  or 
programs.  According  to  Army  life-cycle  guidance,  the  SA3  PM  is  required  to 
obtain  milestone  approval  authority  from  the  Assistant  Secretary  of  the  Army 
(Installations,  Logistics,  and  Environment)  for  any  major  modification  to  an 
existing  system. 

The  objective  of  milestone  approval  at  this  phase  in  a  system's  life  cycle  is  to 
revalidate  whether  the  existing  system  conforms  to  architectural  requirements 
and  whether  the  system  continues  to  satisfy  validated  mission  needs  or  whether 
the  system  should  be  terminated.  This  revalidation  helps  milestone  approval 
officials  to  determine  whether  modifying  the  system  is  the  most  cost- 
effective  alternative. 

To  accomplish  the  milestone  approval,  the  PM  is  required  to  update  all 
life-cycle  documentation,  including  previously  approved  baseline  cost,  schedule, 
and  program  requirements.  Additionally,  the  cost  of  implementing  the 
recommended  modification  should  be  compared  with  other  alternatives.  The 
SA3  PM  did  not  prepare  the  required  updates  for  the  designated  milestone 
approval  official,  and  the  SA3  PM  did  not  consider  other  alternatives  to  the 
M204  conversion.  Without  updated  cost  and  program  requirements  and 
evaluation  of  other  available  alternatives,  the  milestone  approval  official  could 
not  determine  whether  the  M204  conversion  was  the  most  cost- 
effective  solution. 

Modification  Improperly  Based  on  AMC  Direction.  USASAC 
officials  stated  that  alternatives  to  the  SA3  M204  conversion  were  not  evaluated 
because  USASAC  was  directed  by  AMC  to  convert  the  SA3  S2K  format  to  a 
M204  format,  in  accordance  with  AMC  Regulation  18-2,  "AMC  Standard 
Mainframe  Data  Base  Management  Systems,"  November  21,  1988. 

AMC  Regulation  18-2  designated  M204  format  as  the  standard  mainframe  data 
base  management  system  for  use  in  logistics  business  systems  within  AMC. 
However,  the  regulation  was  rescinded  November  29,  1989,  before  USASAC 
initiated  action  to  develop  the  M204  conversion.  Further,  our  discussions  with 
users  indicated  that  SA3  needed  M204  format  capabilities  for  the  system  to  be 
used  at  the  AMC  major  subordinate  commands. 

The  dissatisfaction  with  SA3  at  the  AMC  major  subordinate  commands  was 
confirmed  by  the  results  of  a  USASAC  assessment  completed  in  March  1991. 

Command-Level  Assessment  Confirmed  User  Dissatisfaction.  A  March 
1991  command-level  assessment  of  SA3,  conducted  at  all  six  (now  five)  AMC 
major  subordinate  commands,  confirmed  user  dissatisfaction  with  the  system. 
The  written  assessment  was  prepared  as  a  result  of  input  received  during  site 
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visits  to  the  commands  by  the  USASAC  Director  of  Information  Management 
and  the  SA3  PM.  See  Appendix  E  for  the  full  text  of  the  assessment  of 
automation  to  support  security  assistance  at  AMC  major  subordinate  commands. 
The  assessment  showed  that  SA3  did  not  meet  its  objectives  and  that  system 
users  were  not  satisfied  with  the  initial  software  release.  Specifically,  the 
assessment  stated: 

o  The  Armament,  Munitions,  and  Chemical  Command,  Rock  Island, 
Illinois,  was  the  only  AMC  major  subordinate  command  to  accept  and  use  the 
SA3  International  Logistics  Supply  Delivery  Plan. 

o  The  Aviation  Systems  Command  (now  Aviation  and  Troop 
Command),  St.  Louis,  Missouri,  did  not  transmit  cases  electronically  because  of 
communication  problems.  Case  designs  were  not  standardized  and  many 
manual  changes  were  made  to  cases.  Also,  local  automation  tools  were 
frequently  used. 

o  The  Communications-Electronics  Command,  Fort  Monmouth,  New 
Jersey,  used  SA3  for  most  cases;  however,  the  command  made  limited  use  of 
management  tools  in  SA3. 

o  The  Missile  Command,  Huntsville,  Alabama,  showed  relatively  slow 
acceptance  of  SA3  in  favor  of  locally  developed  tools. 

o  The  Troop  Support  Command  (now  Aviation  and  Troop  Command), 
St.  Louis,  Missouri,  expressed  particular  frustration  with  the  SA3  International 
Logistics  Supply  Delivery  Plan  and  identified  a  specific  need  for  standardized 
case  management  tools. 

o  The  Tank-Automotive  Command,  Warren,  Michigan,  used  SA3  for 
some  case  development,  but  had  old  equipment  and  had  communication 
difficulties  when  transmitting  cases  electronically. 

As  a  whole,  the  assessment  clearly  showed  that  USASAC  and  the  AMC  major 
subordinate  commands  had  received  little  benefit  from  the  $37  million  spent  on 
SA3  by  March  1991. 

USASAC  Actions  Taken  as  a  Result  of  Assessment.  Although  USASAC 
performed  the  assessment  more  than  3  years  ago,  we  found  no  planning 
documents  or  action  plan  to  address  the  problems  noted  and  no  estimate  of  the 
cost  to  correct  the  deficiencies  and  make  SA3  useful  and  effective. 


Adequacy  of  Technical  Support  SIMA  Provided 


SIMA  did  not  follow  established  procedures  or  provide  adequate  technical 
support  during  SA3  development.  Specifically,  SIMA  did  not  always  obtain 
system  change  requests  or  prepare  planning  and  design  documentation  as 
required  before  modifying  and  enhancing  SA3.  In  addition,  SIMA  failed  to 
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perform  key  functions  during  SA3  development  because  SIMA  management,  in 
some  instances,  allowed  the  Security  Assistance  Functional  Support  Office, 
US  AS  AC,  to  perform  duties  that  overlapped  and  interfered  with  SIMA's 
responsibility  as  a  central  design  activity. 

Controls  on  System  Change.  SIMA  performed  development  efforts  on  the 
M204  conversion,  a  major  system  modification  costing  at  least  $3.4  million, 
without  an  approved  system  change  request.  Development  and  Readiness 
Command  [now  Army  Materiel  Command]  Regulation  18-17,  "Automatic  Data 
Processing  Systems  Configuration  Management,"  September  17,  1979,  requires 
a  system  change  request  for  all  taskings  that  involve  a  change  to  an 
information  system. 

SIMA  treated  the  M204  conversion  as  an  internal  project,  bypassing  the  internal 
controls  established  for  system  change  requests.  Internal  projects  do  not  require 
a  system  change  request.  USASAC  submitted  a  system  change  request  for  the 
M204  conversion  that  did  not  include  a  cost-benefit  analysis.  The  system 
change  request  was  appropriately  rejected  by  the  system  change  control  officer 
at  SIMA.  However,  SIMA  bypassed  its  own  control  mechanism  and  developed 
the  M204  conversion  without  an  approved  system  change  request.  As  a  result, 
74,549  of  the  total  111,369  hours  (67  percent)  that  SIMA  charged  to 
SA3  taskings  from  October  1990  through  February  1994  were  performed 
without  SIMA  evaluating  the  alternatives  and  determining  whether  the  proposed 
change  was  necessary  or  cost-effective. 

According  to  SIMA  management,  no  regulations,  policies,  or  guidance  cover 
the  creation  or  use  of  internal  projects.  Further,  SIMA  permitted  lower  level 
management  personnel  such  as  branch  chiefs  or  project  managers  to  authorize 
the  use  of  internal  projects. 

Controls  on  Contractor  Work  Orders.  Over  a  2-year  period  ending  April 
1994,  SIMA  submitted  38  percent  of  its  work  orders  to  the  contractor  through  a 
task  order  contract  without  evaluating  whether  the  planned  work  was  beneficial 
to  the  Government  and  without  maintaining  adequate  oversight  of  the  requested 
actions  in  the  work  orders.  SIMA  bypassed  existing  internal  controls  by 
submitting  work  orders  to  the  contractor  without  first  obtaining  required 
supporting  documentation. 

Contractor  Support  Through  Task  Order  Contract.  SIMA  obtained 
contractor  support  through  a  technical  support  task  order  contract  to: 

o  analyze  system  change  requests  for  maintenance  of  and  enhancements 
to  SA3; 

o  determine  the  impact  of  changes  to  the  operational  system;  and 

o  develop  system  programs,  program  revisions,  and  related 
documentation. 

When  technical  support  is  needed  from  the  contractor,  SIMA  should  submit 
work  orders,  along  with  supporting  documentation,  to  the  contractor. 
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Supporting  Documentation.  Review  of  the  contractor's  list  of 
work  orders  performed  on  technical  support  task  orders,  valued  at  $909,000, 
during  FYs  1993  and  1994  showed  that  SIMA  submitted  46  of  121  work  orders 
(38  percent)  to  the  contractor  without  first  obtaining  the  required  supporting 
documentation.  Either  an  approved  system  change  request  or  a  SIMA 
Form  366a,  "Environmental  System  Test  Report,"  should  be  submitted  with 
each  work  order  to  ensure  the  planned  efforts  are  beneficial  to  the  Government 
and  to  maintain  oversight  of  the  requested  action.  SIMA  Form  366a  is  a  SIMA 
internal  document  to  report,  correct,  and  track  deficiencies  that  occur  during, 
and  as  a  result  of,  pre-installation  testing  within  SIMA. 

Planning  and  Design  Documentation  for  Case  Print  Application.  SIMA  did 
not  adequately  manage  the  development  of  the  SIMA-developed  M204  case 
print  application.  More  precisely,  SIMA  management  allowed  SIMA 
programmers  to  code  (program)  the  case  print  application  without  proper 
planning.  Specifically,  SIMA  did  not: 

o  prepare  functional  descriptions  of  the  requirements, 

o  perform  a  technical  analysis  of  the  requirements,  and 

o  develop  specifications. 

Army  Guidance  Existed  To  Prevent  Misinterpretation.  According  to 
Army  Technical  Bulletin  18-103,  "Army  Automation  Software  Design  and 
Development,"  January  1983,  SIMA  should  have  properly  planned  the  case 
print  application  effort  to  prevent  misinterpretation  and  to  provide  continuity 
between  the  SIMA  and  the  USASAC  functional  proponent.  Further,  one  stated 
objective  of  Army  Technical  Bulletin  18-103  is  to  reduce  the  cost  of 
implementation,  maintenance,  and  modification  of  Army  automated  systems, 
such  as  SA3. 

SEVLA's  Failure  to  Enforce  Existing  Guidance.  Because  SIMA 
management  did  not  enforce  Army  Technical  Bulletin  18-103  and  existing  Army 
guidance,  the  SIMA  programmers  determined  specific  M204  case  print 
requirements  on  their  own  with  minimal  functional  input  from  SIMA 
management  or  USASAC.  SIMA  management  stated  they  agreed  that  the 
procedures  spelled  out  in  Army  Technical  Bulletin  18-103  should  have  been 
followed  for  effective  software  development;  however,  USASAC  controlled  the 
money  needed  to  implement  the  processes  and  had  directed  that  M204  be 
developed  using  USASAC  ideas  regardless  of  the  procedures  in  Army  Technical 
Bulletin  18-103.  SIMA  management  further  stated  that,  because  SIMA  is  a 
"fee-for-service"  organization,  SIMA  had  no  alternative  but  to  adhere  to  the 
USASAC  directions. 

As  a  result  of  the  decision  by  SIMA  management  to  follow  USASAC  direction 
rather  than  Army  guidance,  the  SIMA  programmers  working  on  the  M204  case 
print  program  were  not  provided  with  adequate  guidance  on  the  M204  case  print 
program  requirements.  Therefore,  SIMA  programmers  determined  that  the  best 
approach  for  accomplishing  the  M204  case  print  program  was  to  design  and 
develop  new  software.  The  USASAC  functional  proponent,  on  the  other  hand, 
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DSAA  must  also  assume  responsibility  for  life-cycle  management  of  existing 
Army  security  assistance  information  systems,  such  as  SA3,  to  ensure 
compliance  with  established  life-cycle  management  policies  and  procedures. 

Internal  Controls  on  Funds  Expended.  DSAA,  responsible  for  funding 
SA3  based  on  annual  budget  submissions  from  USASAC  and  SIMA,  did  not 
establish  internal  controls  to  maintain  visibility  over  funds  expended  on  SA3. 
As  a  result,  USASAC  continued  to  fund  SA3  development  and  enhancement, 
ignoring  DSAA  and  other  DoD  guidance  that  imposed  strict  limitations  on 
further  development  and  enhancement  of  security  assistance  and  other  DoD 
information  systems. 

Guidance  Limited  Further  Development.  In  April  1980,  DSAA 
issued  guidance  to  the  Military  Departments  that  indicated  its  intention  to 
actively  participate  in  the  early  review  of  planned  information  system 
development  and  maintenance  before  approving  related  requests  for  funds.  The 
guidance  required  that  all  information  system  projects  with  development  and 
investment  costs  of  $100,000  or  more,  or  estimated  annual  operating  and 
maintenance  costs  of  $200,000  or  more,  or  both  be  submitted  to  DSAA  for 
approval  before  inclusion  in  the  annual  budget. 

In  addition,  the  Joint  Logistics  Systems  Center,  established  February  11,  1992, 
required  that  no  further  development  and  only  essential  maintenance  should  be 
performed  on  existing  logistics  information  systems  that  were  not  selected  as 
standard  DoD  systems.  The  Joint  Logistics  Systems  Center  was  created  in 
response  to  a  DoD  CIM  initiative  to  standardize  DoD  automated  information 
systems.  (See  Part  I,  Other  Matters  of  Interest  for  details  on  the  initiative.) 

USASAC  Continued  Development  on  SA3.  Although  DSAA  denied 
numerous  USASAC  and  SIMA  budget  requests  to  develop  and  enhance  SA3, 
the  Army  continued  to  develop  and  enhance  SA3.  USASAC  and  SIMA 
accomplished  this  work  on  projects  that  were  previously  denied  funding  by 
using  excess  funds  available  from  prior  years  and  by  using  funds  approved  by 
DSAA  to  accomplish  other  specific  projects.  Further,  of  the  total  funds 
provided  to  USASAC  and  SIMA,  neither  activity  could  provide  a  breakdown  of 
development  costs  versus  maintenance  costs.  The  following  examples 
demonstrate  how  USASAC  and  SIMA  continued  funding  projects  to  develop 
and  enhance  SA3  without  DSAA  approval. 

o  Over  the  past  6  years,  USASAC  consistently  funded  work  years  for 
SA3  at  a  significantly  high  level  despite  USASAC  claims  that  SA3  is  considered 
fully  developed  and  that  budget  requests  are  only  for  system  maintenance. 

In  FY  1989,  when  the  system  was  still  under  full  development,  USASAC 
funded  49  work  years  of  effort.  However,  in  FY  1994,  5  years  later,  USASAC 
funded  44  work  years  of  effort  even  though  SA3  was  supposed  to  be  fully 
developed  in  FY  1990.  The  44  work  years  funded  in  FY  1994  do  not  include 
the  approximate  24  work  years  of  effort  designated  for  automation  support  at 
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depended  on  the  Security  Assistance  Functional  Support  Office  personnel,  who 
had  knowledge  of  security  assistance  functions,  but  had  little  or  no  skills  to 
resolve  technical  problems. 

As  a  result  of  the  inadequate  test  and  evaluation,  SIMA  accepted  and  paid  for  at 
least  two  minor  software  applications  (see  Figure  2),  valued  at  about  $160,000, 
without  adequate  testing  and  that  users  never  used.  For  example,  case  batch 
processing  was  a  contractor-developed  application  to  allow  the  transfer  of  data 
from  an  outside  source  into  the  S2K  data  base  format.  At  the  time  of 
development,  no  software  was  available  to  provide  sources  of  data  necessary  for 
testing  this  application.  USASAC  accepted  the  application,  even  though  data 
was  not  available  to  fully  test  it. 

Inadequate  Customer  Support.  SIMA  did  not  provide  adequate 
customer  support  for  SA3.  As  a  central  design  activity,  SIMA  is  responsible 
for  providing  user  assistance,  advice,  and  guidance  during  all  phases  of  standard 
systems  life-cycle  processing.  USASAC  officials  stated  that  because  SIMA  did 
not  perform  most  of  the  SA3  division-level  testing,  SIMA  lacked  required 
knowledge  on  SA3  applications.  USASAC  officials  also  stated  that  SIMA  did 
not  provide  adequate  assistance  to  the  users  who  were  experiencing  problems 
with  SA3  applications.  To  keep  SA3  properly  functioning,  the  Security 
Assistance  Functional  Support  Office  personnel,  not  SIMA  personnel, 
responded  to  user  problems  and  questions. 

The  lack  of  direct  communication  between  SIMA  and  its  customers  may  have 
impeded  its  ability  to  correct  identified  problems  and  deficiencies.  See 
examples  of  the  problems  with  the  SIMA-developed  case  print  application  in 
Appendix  C.  As  a  result  of  our  audit,  both  SIMA  and  USASAC  took  action  to 
further  define  their  responsibilities.  For  example,  user  problems  and  concerns 
are  now  directly  communicated  to  and  resolved  by  SIMA.  SA3  users  at  the 
AMC  major  subordinate  commands  indicated  that  problems  were  quickly  and 
effectively  resolved  when  they  dealt  directly  with  SIMA  programmers. 

Contractor  Oversight.  SIMA  did  not  provide  adequate  oversight  of 
contractor  performance  on  the  personal  computer  case  print  application.  As  a 
central  design  activity,  SIMA  is  responsible  for  initially  deciding  whether  a 
proposed  task  will  be  performed  using  SIMA  or  contractor  personnel.  When 
the  contractor  is  selected  to  perform  the  task,  SIMA  is  responsible  for 
developing  the  statement  of  work  and  monitoring  contractor  performance.  In 
this  instance,  however,  the  Security  Assistance  Functional  Support  Office 
improperly  directed  that  SIMA  task  the  contractor  to  develop  the  personal 
computer  case  print  application.  In  addition,  SIMA  prepared  the  statement  of 
work  without  adequate  review  or  input  from  SIMA  programmers. 

Requirement  Not  Clearly  Defined.  SIMA  did  not  adequately 
respond  to  strong  indications  from  the  contractor  that  the  contractor 
misunderstood  the  overall  requirement  for  the  personal  computer  case  print 
application.  The  requirement  for  the  personal  computer  case  print  application 
was  not  clearly  defined,  as  evidenced  by  the  contractor's  bid  of  $160,000  to 
develop  the  personal  computer  case  print  application,  compared  to  the 
GSA  estimate  of  $37,413.  SIMA  did  not  develop  its  own  estimate  or  review 
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and  otherwise  approve  either  the  contractor  bid  or  the  GSA  estimate. 
Consequently,  GSA  negotiated  a  final  price  of  $80,182  (including  a 
13.9  percent  GSA  surcharge)  for  the  contractor  to  develop  the  personal 
computer  case  print  application.  Appendix  A  provides  further  details  on  the 
GSA  role  in  procuring  SA3  software  applications. 

Before  GSA  awarded  the  ceiling  price  task  order  to  the  contractor,  the  SIMA 
client  representative  noted  in  a  memorandum  to  GSA: 

I  do  not  agree  with  the  total  amount  of  funding  shown  for  this 
task  ....  there  is  no  way  this  total  analysis  program  effort  can  be 
developed  for  $80,000,  even  if  GSA  cost  was  not  included.  ...  I 
don't  know  what  (the  contractor)  submitted  to  you  for  their  estimate, 
however,  if  theirs  was  in  the  amount  on  the  agreement,  I  don't  believe 
the  overall  requirements  were  clearly  understood. 

Even  with  the  comments  from  its  own  client  representative  that  the  requirements 
were  not  clearly  understood  by  both  parties,  SIMA  took  no  action  to  clarify 
the  requirements. 

Consequence  of  Unclear  Requirements.  Because  SIMA  did  not 
clarify  the  requirements  for  the  personal  computer  case  print  application,  the 
Army  spent  $73,852  on  the  application  without  receiving  an  application  that 
could  be  used  to  print  FMS  cases.  A  ceiling-price  task  order  was  issued  that 
required  the  contractor  to  develop  the  personal  computer  case  print  application 
by  accomplishing  the  following  three  tasks: 

o  develop  a  technical  system  design  document; 

o  code,  test,  and  debug  the  software  application;  and 

o  perform  training. 

However,  after  incurring  costs  of  $73,852  (including  the  GSA  surcharge)  and 
only  completing  the  first  task,  the  contractor  reported  that  the  application  could 
not  be  completed  on  schedule.  SIMA  canceled  the  task  order  and  stated  that 
GSA  provided  inefficient  contract  administration.  Although  the  requirement 
still  existed,  neither  SIMA  nor  USASAC  took  action  to  finish  the  personal 
computer  case  print  application. 


Adequacy  of  Army  Management  Oversight  of  SA3 


The  Assistant  Secretary  of  the  Army  (Installations,  Logistics,  and 
Environment),  as  the  responsible  information  systems  approval  authority  within 
the  Army,  did  not  approve  SA3  at  critical  milestones,  did  not  ensure  that 
development  efforts  were  accomplished  within  approved  milestones,  and  did  not 
ensure  that  SA3  met  mission  and  user  needs.  AMC  supplemented  the  approval 
authority  that  existed  at  the  Assistant  Secretary-level  by  establishing  the 
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Logistical  Systems  Review  Committee  (the  Committee)  to  review  and  approve 
development  and  modifications  and  provide  mission  analysis  on  all  AMC 
information  systems. 

The  Committee  did  not  obtain  and  review  required  life-cycle  management 
documentation.  Figure  4  clearly  shows  that  updated  cost  and  program 
requirements  could  have  helped  Army  management  to  determine  whether  it  was 
more  cost-  and  mission-effective  to  continue  with  SA3  development  as  planned 
or  to  terminate,  or  possibly  redirect,  ongoing  and  planned  development  efforts. 
In  addition,  the  Committee  did  not  verify  that  SA3  met  mission  and  user  needs 
once  the  system  was  deployed. 

We  attribute  Army  management's  apparent  lack  of  interest  in  SA3  to  the  fact 
that  SA3  was  funded  using  Defense  Security  Assistance  Agency  (DSAA)  rather 
than  Army  funds. 

Although  Army  management  did  not  perform  the  required  oversight  of  SA3,  we 
made  no  recommendations  to  the  Army  because  the  Joint  Logistics  Systems 
Center  is  now  responsible  for  approving  the  development  and  modification  of 
logistics  information  systems,  including  SA3.  (See  Part  I,  Other  Matters 
of  Interest.) 


Adequacy  of  DSAA  Oversight  of  SA3  Funding 


DSAA  did  not  adequately  oversee  SA3  life-cycle  management  responsibilities 
that  were  delegated  to  the  Army.  In  addition,  DSAA  did  not  effectively 
monitor  the  funds  provided  to  the  Army  for  SA3  development  and  maintenance. 
As  a  result,  US  AS  AC  continues  to  spend  as  much  as  $3.7  million  annually  to 
upgrade  and  enhance  SA3  without  a  current  management  plan.  Further,  DSAA 
allowed  USASAC  to  use  the  money  to  fund  developmental  efforts  that  were 
never  approved  or  that  were  previously  denied  funding  by  DSAA. 

Internal  Controls  on  Life-Cycle  Management.  DSAA  did  not  establish 
internal  controls  to  verify  that  the  Army  complied  with  existing  life-cycle 
management  regulations  when  developing  and  maintaining  information  systems 
funded  by  DSAA.  For  SA3  development  and  maintenance,  DSAA  relied  on  the 
Army  to  follow  established  DoD  and  internal  Army  guidance  on  life-cycle 
management.  However,  as  discussed  previously,  Army  officials  did  not 
perform  required  life-cycle  management  reviews  of  SA3  or  provide  critical 
system  milestone  approval.  In  addition,  Army  officials  are  no  longer 
responsible  for  approving  development  and  modification  efforts  on  Army 
logistics  information  systems,  such  as  SA3. 

DSAA  recently  requested  approval  from  the  Assistant  Secretary  of  Defense 
(Command,  Control,  Communications,  and  Intelligence)  to  initiate  and  oversee 
plans  to  develop  a  standard  security  assistance  information  system.  If  DSAA  is 
delegated  this  oversight  responsibility  for  new  information  system  development, 
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DSAA  must  also  assume  responsibility  for  life-cycle  management  of  existing 
Army  security  assistance  information  systems,  such  as  SA3,  to  ensure 
compliance  with  established  life-cycle  management  policies  and  procedures. 

Internal  Controls  on  Funds  Expended.  DSAA,  responsible  for  funding 
SA3  based  on  annual  budget  submissions  from  US  AS  AC  and  SIMA,  did  not 
establish  internal  controls  to  maintain  visibility  over  funds  expended  on  SA3. 
As  a  result,  USASAC  continued  to  fund  SA3  development  and  enhancement, 
ignoring  DSAA  and  other  DoD  guidance  that  imposed  strict  limitations  on 
further  development  and  enhancement  of  security  assistance  and  other  DoD 
information  systems. 

Guidance  Limited  Further  Development.  In  April  1980,  DSAA 
issued  guidance  to  the  Military  Departments  that  indicated  its  intention  to 
actively  participate  in  the  early  review  of  planned  information  system 
development  and  maintenance  before  approving  related  requests  for  funds.  The 
guidance  required  that  all  information  system  projects  with  development  and 
investment  costs  of  $100,000  or  more,  or  estimated  annual  operating  and 
maintenance  costs  of  $200,000  or  more,  or  both  be  submitted  to  DSAA  for 
approval  before  inclusion  in  the  annual  budget. 

In  addition,  the  Joint  Logistics  Systems  Center,  established  February  11,  1992, 
required  that  no  further  development  and  only  essential  maintenance  should  be 
performed  on  existing  logistics  information  systems  that  were  not  selected  as 
standard  DoD  systems.  The  Joint  Logistics  Systems  Center  was  created  in 
response  to  a  DoD  CIM  initiative  to  standardize  DoD  automated  information 
systems.  (See  Part  I,  Other  Matters  of  Interest  for  details  on  the  initiative.) 

USASAC  Continued  Development  on  SA3.  Although  DSAA  denied 
numerous  USASAC  and  SIMA  budget  requests  to  develop  and  enhance  SA3, 
the  Army  continued  to  develop  and  enhance  SA3.  USASAC  and  SIMA 
accomplished  this  work  on  projects  that  were  previously  denied  funding  by 
using  excess  funds  available  from  prior  years  and  by  using  funds  approved  by 
DSAA  to  accomplish  other  specific  projects.  Further,  of  the  total  funds 
provided  to  USASAC  and  SIMA,  neither  activity  could  provide  a  breakdown  of 
development  costs  versus  maintenance  costs.  The  following  examples 
demonstrate  how  USASAC  and  SIMA  continued  funding  projects  to  develop 
and  enhance  SA3  without  DSAA  approval. 

o  Over  the  past  6  years,  USASAC  consistently  funded  work  years  for 
SA3  at  a  significantly  high  level  despite  USASAC  claims  that  SA3  is  considered 
fully  developed  and  that  budget  requests  are  only  for  system  maintenance. 

In  FY  1989,  when  the  system  was  still  under  full  development,  USASAC 
funded  49  work  years  of  effort.  However,  in  FY  1994,  5  years  later,  USASAC 
funded  44  work  years  of  effort  even  though  SA3  was  supposed  to  be  fully 
developed  in  FY  1990.  The  44  work  years  funded  in  FY  1994  do  not  include 
the  approximate  24  work  years  of  effort  designated  for  automation  support  at 
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the  AMC  major  subordinate  commands  during  the  same  period.  The  work 
years  funded  also  do  not  include  contractor  support  obtained  for  internal  use  by 
the  commands. 

o  From  FYs  1988  through  1990,  USASAC  funded  GSA  $2.8  million  to 
develop  a  statement  of  work  for  the  integration  of  SA3  (GSA  integration 
project).  On  May  31,  1991,  USASAC  canceled  the  project  and  transferred  the 
remaining  balance  of  $2.7  million  to  continued  funding  maintenance  and 
development  efforts,  abandoning  SA3  integration.  Integration  with  the  Army 
Commodity  Command  Standard  System  is  a  primary  objective  for  SA3  and  was 
one  of  the  major  deficiencies  noted  by  SA3  users.  USASAC  continues  to 
address  the  importance  of  SA3  integration  and,  in  FY  1991,  reported  to  DSAA 
that  an  internal  initiative  would  streamline  operations  and  fully  integrate  SA3. 
Integration  of  SA3  was  not  achieved  because  USASAC  has  continued  to  fund 
SA3  maintenance  and  other  development  efforts  at  the  expense  of  integration. 

o  USASAC  funded  the  Missile  Command  at  least  $651,000  over  3  years 
to  develop  the  Security  Assistance  Management  Acquisition  Program 
application.  This  application  was  never  referred  to  in  any  budget  documents 
and,  therefore,  was  never  approved  by  DSAA.  The  application  was  to  replace 
the  International  Logistics  Supply  Delivery  Plan,  a  major  SA3  systems 
application  that  was  fielded  and  never  used. 

Best  Position  for  Oversight.  Although  SA3  falls  within  Joint  Logistics 
Systems  Center  cognizance,  the  Joint  Logistics  Systems  Center  does  not  have 
visibility  over  SA3  funding.  Because  SA3  is  funded  by  DSAA,  DSAA  is  in  the 
best  position  to  provide  oversight  of  SA3  and  to  enforce  accountability  of  the 
funds  provided  for  SA3.  This  oversight  should  include  site  visits  by  DSAA  to 
the  AMC  major  subordinate  commands  to  determine  what  resources  are 
necessary  to  manage  the  Security  Assistance  Program  through  automation. 


Conclusion 


To  avoid  costly  and  duplicative  information  system  development  for  the 
Security  Assistance  Program,  DSAA  should  discontinue  further  funding  for 
SA3  development  and  enhancement,  in  accordance  with  current  DoD  CIM  and 
other  DoD  initiatives.  In  addition,  to  ensure  that  unjustified  development  is  not 
performed,  DSAA  should  withhold  funds  for  SA3  maintenance  until  the  Army 
identifies  and  justifies  only  essential  software  and  hardware  maintenance  costs. 

Without  updated  overall  planning  documents  or  an  estimated  cost  to  complete 
the  system,  USASAC  was  forced  to  develop  SA3  on  a  piece-meal  basis.  In 
addition,  because  USASAC  did  not  establish  a  sound  project  baseline,  including 
reliable  cost  and  program  requirements,  SA3  could  not  be  developed  in  a  cost- 
effective  manner  or  meet  its  intended  objective.  In  the  final  analysis,  SA3  did 
not  provide  the  Army  with  a  highly  responsive  and  flexible  automated  system 
necessary  for  total  management  of  the  Security  Assistance  Program. 
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Unless  required  documentation  and  cost  estimates  are  properly  prepared  with 
current,  accurate,  and  complete  information,  management  cannot  ensure  that: 

o  available  resources  are  used  to  provide  the  most  beneficial  results, 

o  expected  benefits  are  worth  projected  costs,  and 

o  the  lowest  cost  approach  is  selected  and  will  be  followed  during 
system  development  and  use. 

The  Army  needs  to  ensure  that  required  life-cycle  management  policies  and 
procedures  are  followed.  DSAA  needs  to  develop  effective  oversight  of  funding 
provided  to  the  Army  to  maintain  SA3. 

Further,  SIMA  has  not  provided  adequate  technical  support  of  SA3  development 
and  enhancement.  Problems  with  the  development  and  enhancement  of 
software  applications  will  continue  unless  more  rigorous  software  maintenance 
processes  are  followed  at  SEMA. 


Management  Comments  on  the  Finding  and  Audit  Response 


Although  DSAA  did  not  comment  on  the  finding,  the  Army  commented 
extensively.  See  Appendix  F  for  a  summary  of  the  Army  comments  and  the 
audit  response. 


Recommendations,  Management  Comments,  and  Audit 
Response 


Revised  Recommendations.  Based  on  DSAA  comments,  we  revised 
Recommendation  l.c.  to  perform  site  visits  at  the  AMC  major  subordinate 
commands  to  determine  resources  necessary  to  manage  the  Security  Assistance 
Program.  Based  on  Army  comments,  we  modified  draft  report 
Recommendation  3.b.,  eliminating  reference  to  limitations  on  internal  projects. 

1.  We  recommend  that  the  Director,  Defense  Security  Assistance  Agency: 

a.  Discontinue  funding  for  the  future  development  of  the  Security 
Assistance  Automation,  Army,  information  system  in  accordance  with  Joint 
Logistics  Systems  Center  policies. 

b.  Withhold  funding  for  future  maintenance  of  the  Security 
Assistance  Automation,  Army,  information  system  until  the  U.S.  Army 
Security  Assistance  Command  identifies  and  justifies  only  essential 
maintenance  expenditures  of  software  and  hardware  requirements  for  the 
Security  Assistance  Automation,  Army,  information  system  in  accordance 
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with  DoD  Instruction  8120.2,  "Automated  Information  Systems  Life-Cycle 
Management  Process,  Review,  and  Milestone  Approval  Procedures,"  and 
Army  Regulation  25-3,  "Life-Cycle  Management  of  Information  Systems." 

c.  Perform  site  visits  at  the  Army  Materiel  Command  major 
subordinate  commands  to  determine  resources  necessary  to  manage  the 
Security  Assistance  Program  through  automation. 

d.  Establish  internal  control  procedures  to  verify  that  the  U.S. 
Army  Security  Assistance  Command  adheres  to  DoD  and  Army  life-cycle 
management  regulations  and  accounts  for  funds  expended  on  the  Security 
Assistance  Automation,  Army,  information  system. 

Management  Comments.  DSAA  concurred  with  Recommendations  l.a.,  l.b., 
and  l.d.,  stating  that,  during  the  FY  1995  budget  process,  DSAA  disapproved 
funding  for  all  but  maintenance  of  SA3  and  that  Army  funding  for  SA3  has 
been  reduced  to  only  essential  maintenance.  In  addition,  within  90  days  of 
finalizing  the  audit  report,  DSAA  will  require  the  Army  to  provide  quarterly 
reports  on  the  life-cycle  management  and  status  of  funds  for  SA3.  DSAA 
concurred  with  the  intent  of  draft  report  Recommendation  l.c.,  but  stated  that 
the  responsibility  for  identifying  and  reviewing  alternatives  and  costs  for  system 
development  and  identifying  and  meeting  user  needs  is  a  Department  of  the 
Army  responsibility. 

Audit  Response.  Based  on  management  comments,  we  revised 
Recommendation  l.c.  Management  did  not  comment  on  the  potential  monetary 
benefits  associated  with  Recommendation  l.a.  We  ask  that  management 
provide  those  comments  and  comment  on  the  revised  recommendation  in 
response  to  the  final  report. 

2.  We  recommend  that  the  Commander,  U.S.  Army  Security 
Assistance  Command,  Army  Materiel  Command: 

a.  Establish  procedures  to  verify  that  approval  of  software  changes 
for  the  Security  Assistance  Automation,  Army,  information  system  and 
that  tracking,  updating,  and  reporting  of  costs  associated  with  the  system 
comply  with  Army  Regulation  25-3,  "Life-Cycle  Management  of 
Information  Systems." 

b.  Periodically  review  the  Security  Assistance  Automation,  Army, 
information  system  as  a  control  technique  to  determine  compliance  with 
DoD  Instruction  8120.2,  "Automated  Information  Systems  Life-Cycle 
Management  Process,  Review,  and  Milestone  Approval  Procedures,"  and 
Army  Regulation  25-3,  "Life-Cycle  Management  of  Information  Systems." 

c.  Evaluate  the  Security  Assistance  Automation,  Army,  information 
system  to  identify  total  user  software  and  hardware  requirements, 
including  all  personnel  required  to  support  the  system. 
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d.  Based  on  the  evaluation  of  the  Security  Assistance  Automation, 
Army,  information  system  performed  in  Recommendation  2.c.,  determine 
the  estimated  cost  to  complete  the  system,  the  estimated  cost  to  maintain 
the  system,  and  the  expected  benefits  and  cost  reductions  that  will  be 
achieved  when  the  system  is  completed.  This  information  should  be 
included  as  part  of  the  command's  annual  budget  submission  to  the 
Defense  Security  Assistance  Agency. 

e.  Disestablish  the  Security  Assistance  Functional  Support  Office, 
St.  Louis,  Missouri,  as  it  adds  no  value  to  the  Security  Assistance 
Automation,  Army,  information  system  program  function,  and  develop 
functions  that  can  be  adequately  provided  by  the  Systems  Integration  and 
Management  Activity,  Army  Materiel  Command. 

Management  Comments.  The  Army  concurred  with  our  recommendations, 
stating  that  a  senior-level  review  committee  will  be  established  to  provide 
oversight  of  security  assistance  information  systems  and  that  the  committee  will 
assess  compliance  semiannually,  beginning  in  July  1995.  The  Army  also  stated 
that  it  will  prepare  a  business  case  and  a  functional  economic  analysis  to 
evaluate  SA3  and  that,  based  on  the  evaluation,  the  Army  will  provide  estimated 
costs  and  expected  benefits  of  the  SA3  information  system  to  DSAA  as  part  of 
the  command's  annual  budget  submission,  beginning  in  August  1995.  In 
addition,  the  Army  stated  that  action  will  be  taken  to  disestablish  the  functional 
support  office  by  October  1995. 

Audit  Response.  The  Army  comments  were  responsive. 

3.  We  recommend  that  the  Director,  Systems  Integration  and 
Management  Activity,  Army  Materiel  Command: 

a.  Develop  additional  internal  control  objectives  and  techniques  on 
the  documentation  requirements  for  the  Security  Assistance  Automation, 
Army,  information  system. 

b.  Establish  procedures  on  the  use  of  internal  projects. 

c.  Develop  and  issue  policy  prohibiting  the  acceptance  of  work  to 
enhance  or  modify  the  Security  Assistance  Automation,  Army,  information 
system  unless  a  system  change  request  accompanies  the  work  request. 

d.  Develop  and  issue  policy  discontinuing  the  practice  of  submitting 
work  orders  for  Security  Assistance  Automation,  Army,  information 
system  to  the  contractor  without  required  supporting  documentation. 

e.  Create  a  formal  plan,  including  design  and  testing  requirements, 
before  performing  any  additional  Security  Assistance  Automation,  Army, 
information  system  development,  maintenance,  or  modification. 

Management  Comments.  The.  Army  fully  concurred  with  Recom¬ 
mendations  3.  a.,  3.C.,  3.d.,  and  3.e.  and  partially  concurred  with 

Recommendation  3.b.  The  Army  stated  that  it  will  develop  additional  internal 


31 


Management  of  Security  Assistance  Automation,  Army 


control  objectives  and  techniques  on  the  documentation  requirement  for 
SA3  and  establish  procedures  on  the  use  of  internal  projects.  The  Army  agreed 
to  issue  policy  statements  that  a  system  change  request  is  required  before 
accepting  work  to  modify  and  enhance  SA3  and  that  supporting  documentation 
is  required  for  all  work  orders  submitted  to  contractors.  The  Army  also  agreed 
to  create  a  formal  project  plan  before  performing  additional  SA3  development, 
modification,  or  maintenance.  The  Army  nonconcured  with  the  part  of  draft 
report  Recommendation  3.b.  that  limited  the  use  of  internal  projects  to 
administrative  support  functions.  The  Army  stated  that  projects  other  than 
administrative  support  functions  are  managed  as  internal  projects.  The  Army 
provided  specific  dates  of  completion  for  each  planned  action  and  estimated  the 
completion  of  all  planned  actions  by  March  31,  1995. 

Audit  Response.  Management  comments  were  responsive.  The  Army's  plan 
to  establish  procedures  on  the  use  of  internal  projects  meets  the  intent  of  our 
recommendation. 
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Appendix  A.  Additional  Security  Assistance 
Automation,  Army,  Background 


The  primary  objective  of  SA3  was  to  provide  the  necessary  software,  hardware, 
and  communications  to  fully  support  the  Army  Security  Assistance  Program  and 
FMS  case  management  through  automation.  SA3  was  expected  to  serve  about 
1 ,455  potential  users  and  to  link  all  organizations  supporting  the  Security 
Assistance  Program,  including: 

o  Office  of  the  Secretary  of  Defense, 
o  Department  of  the  Army, 
o  AMC  and  its  major  subordinate  commands, 
o  DSAA,  and 
o  GSA. 

The  development  SA3  was  financed  through  the  FMS  Trust  Fund.  From 
FYs  1983  through  1994,  USASAC  spent  more  than  $46  million.  Since 
FY  1992,  USASAC  continued  to  spend  a  relatively  consistent  amount  per  year 
on  SA3.  The  table  below  shows  SA3  costs  from  the  inception  of  the  program  in 
FY  1983  through  FY  1994. 


SA3  Costs  Through  FY  1994 

FY 

Annual  Cost 

Cumulative  Cost 

1983-1988 

$22,803,000 

1989 

5,504,000 

$28,307,000 

1990 

6,550,000 

34,857,000 

1991 

4,015,000 

38,872,000 

1992 

2,614,000 

41,486,000 

1993 

2,375,000 

43,861,000 

1994 

2,836,000 

46,697,000 

Information  System  Development  Project  Management.  The  then-Assistant 
Secretary  of  the  Army  (Installations,  Logistics,  and  Financial  Management) 
approved  the  SA3  PM's  charter  in  1983.  The  charter  placed  the  SA3  PM  under 
the  supervision  and  control  of  the  Commander,  USASAC,  and  also  required 
that  the  SA3  PM  report  to  executive  management  within  the  Department  of  the 
Army  headquarters  and  AMC  on  the  overall  status  of  SA3.  Also,  in  1983, 
USASAC  established  the  Security  Assistance  Functional  Support  Office, 
St.  Louis,  Missouri,  to  provide  customer  support  and  perform  the  liaison 
functions  between  SIMA  and  users  at  AMC  major  subordinate  commands. 

Technical  Support  for  Information  System  Design,  Development,  and 
Maintenance.  SIMA,  a  subordinate  activity  under  AMC,  is  responsible  for  the 
design,  development,  and  maintenance  of  information  systems  that  support 
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AMC.  The  Army  Commodity  Command  Standard  System,  the  standard 
information  system  in  AMC,  is  designed  to  provide  uniform  application  of 
logistics  policy  throughout  the  Army  supply  system.  SA3  was  expected  to  be 
integrated  with  the  Army  Commodity  Command  Standard  System  to  support  the 
FMS  mission.  SIMA  uses  in-house  and  contractor  personnel  to  support  its 
mission.  Contract  support  is  provided  through  task  order  contracts  issued  and 
administered  by  GSA. 

GSA  Role  in  SA3  Procurements.  A  memorandum  of  understanding 
establishes  an  interagency  agreement  between  GSA  and  SIMA  (the  client). 
Through  the  Federal  Information  Systems  Support  Program,  GSA  provides 
information  processing  services  and  contract  administration  services  to  various 
clients  (including  SIMA).  When  SIMA  requires  contractor  support  to 
supplement  its  in-house  development  efforts,  SIMA  submits  a  statement  of  work 
to  GSA.  GSA  issues  a  task  order  with  the  statement  of  work  to  the  contractor. 
GSA  provides  these  services  for  a  fee,  which  is  currently  a  13.9  percent 
surcharge  added  to  the  cost  of  the  task  order. 


35 


Appendix  B.  Summary  of  Allegations  and 
Audit  Results 


Allegation  1.  A  fixed-price  contract  was  awarded  to  a  contractor  to 
develop  application  760  (case  batch  processing).  The  contractor  received 
full  payment  even  though  the  application  was  not  usable  by  the 
Government. 

Audit  Results.  Allegation  1  is  substantiated.  Application  760  was  developed 
and  tested,  the  contractor  received  full  payment  of  $74,800,  and  the  application 
was  fielded  to  end  users.  Audit  results  indicate  that  the  application  was 
never  used. 

Allegation  2.  Government  employees  (Mr.  *  and  Mr.  *,  USASAC,  and 
Mr.  *,  SIMA),  who  were  responsible  for  determining  whether  the  terms  of 
the  contract  were  complete,  participated  in  the  process  to  pay  the  contract 
with  full  knowledge  that  application  760  (case  batch  processing)  would  be 
unusable  to  the  Government. 

Audit  Results.  Allegation  2  is  not  substantiated.  We  did  not  substantiate  that 
Government  employees  had  any  knowledge  that  application  760  would  be 
unusable.  The  application  was  accepted  with  the  understanding  that  it  could  not 
be  fully  tested  until  application  1404  was  developed  (see  allegation  3). 

Allegation  3.  A  fixed-price  contract  was  awarded  to  a  contractor  to 
develop  application  1404  (UNISYS  case  transfer).  The  contractor  received 
full  payment  even  though  the  application  was  not  usable  by  the 
Government. 

Audit  Results.  Allegation  3  is  substantiated.  The  contractor  received  full 
payment  of  $85,400  for  developing  application  1404  under  a  fixed-price 
contract.  USASAC  canceled  the  requirement  for  this  application  before 
full  testing.  As  a  result,  the  application  was  not  used. 

Allegation  4.  Government  employees  (Mr.  *  and  Mr.  *,  USASAC,  and 
Mr.  *,  SIMA),  who  were  responsible  for  determining  whether  the  terms  of 
the  contract  were  complete,  participated  in  the  process  to  pay  the  contract 
with  full  knowledge  that  application  1404  (UNISYS  case  transfer)  would  be 
unusable  to  the  Government. 

Audit  Results.  Allegation  4  is  partially  substantiated.  Government  employees 
accepted  the  application  after  it  was  developed.  The  contractor  did  receive  full 
payment.  The  application  was  then  canceled  before  testing.  We  could  not 
substantiate  whether  the  Government  employees  had  "full  knowledge". 


^Privacy  Act  information  deleted. 
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Allegation  5.  A  ceiling-price  contract  was  awarded  to  a  contractor  to 
develop  a  personal  computer  case  print  application.  The  scope  of  the 
contract  was  changed  to  make  work  for  the  contractor,  work  on  the 
contract  has  been  completed  by  Government  employees,  and  the  contractor 
is  still  billing  hours  against  the  contract,  even  though  the  contract  has  been 
completed  by  Government  employees. 

Audit  Results.  Allegation  5  is  partially  substantiated.  A  ceiling-price  contract 
was  awarded  to  the  contractor  to  develop  the  personal  computer  case  print 
application.  The  scope  of  the  contract  was  not  changed.  While  the  contractor 
was  developing  the  personal  computer  case  print  application,  SIMA  employees 
were  developing  a  program  called  M204  case  print.  Audit  results  indicate  that 
this  SIMA  effort  also  could  have  produced  a  personal  computer  case  print 
program.  We  found  no  evidence  that  the  contractor  submitted  false  billings. 

Allegation  6.  Two  USAS  AC  employees,  Mr.  *  and  Mr.  *,  have  engaged  in 
making  work  for  the  contractors. 

Audit  Results.  Allegation  6  is  not  substantiated.  Poor  management  by 
USASAC  and  SIMA  contributed  to  the  perception  of  making  work  for  the 
contractor.  The  three  applications  mentioned  in  allegations  1  through  5  are 
examples  of  poor  management  that  resulted  in  the  perception  of  making  work 
for  the  contractor. 

Allegation  7.  Two  USASAC  employees,  Mr.  *  and  Mr.  *,  have  continued 
to  issue  contracts  for  the  development  of  useless  computer  programs  that 
have  been  fielded  to  the  end  users,  never  or  seldom  used,  and  subsequently 
deleted. 

Audit  Results.  Allegation  7  is  partially  substantiated.  USASAC  does  not  issue 
contracts.  Contracts  are  issued  by  GSA  through  an  interagency  agreement  with 
SIMA.  Three  applications  named  in  the  allegations  were  awarded  to  the 
contractor  on  task  orders.  One  application  (760)  was  developed,  fielded,  and 
never  used.  USASAC  canceled  the  second  (1404)  before  it  was  fielded.  The 
SIMA  canceled  the  third  (personal  computer  case  print)  before  its  completion. 
The  contractor  received  full  payment  for  each  of  these  task  orders. 
Additionally,  audit  results  indicated  that  several  other  applications,  because  of 
mismanagement,  were  unusable  by  the  Government  (for  example,  the 
International  Logistics  Supply  Delivery  Plan  and  the  Security  Assistance 
Management  Acquisition  Program).  See  the  finding  in  Part  II  for  details. 

Allegation  8.  Three  USASAC  employees,  Mr.  *,  Mr.  *,  and  Mr.  *,  have 
engaged  the  contract  programmers  for  personal  services. 

Audit  Results.  Allegation  8  is  not  substantiated.  Audit  results  were 
inconclusive  in  determining  whether  USASAC  employees  used  contract 
employees  to  perform  personal  services  (see  allegation  9). 
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Allegation  9.  A  SIMA  employee,  Mr.  **,  has  used  contractors  to  perform 
personal  services,  such  as  typing,  flow  chart  development,  and  other 
secretarial  services. 

Audit  Results.  Allegation  9  is  not  substantiated.  No  documents  supported  the 
allegation  that  Mr.  *  or  SIMA  used  the  contractor  to  perform  personal  services. 
However,  in  July  1993,  GSA  representatives  visited  SIMA  to  discuss 
regulations  for  avoiding  personal  services.  Two  GSA  information  technology 
managers  had  alleged  perceptions  of  personal  services  and  other  irregularities  at 
SIMA.  During  this  visit,  GSA  provided  a  new  memorandum  of  understanding 
to  SIMA,  re-establishing  the  understanding  that  SIMA  will  go  through  GSA 
when  contracting  for  computer  software  services.  GSA  also  gave  SIMA  a  copy 
of  the  Federal  Information  Systems  Support  Program  guidelines  on  avoiding 
personal  services. 

Allegation  10.  A  SIMA  employee,  Mr.  *,  has  used  contractor  employees  to 
prepare  statements  of  work  that  have  been  subsequently  awarded  to 
this  contractor. 

Audit  Results.  Allegation  10  is  not  substantiated.  Audit  results  did  not 
indicate  that  this  employee  or  SIMA  used  the  contractor  to  write  statements 
of  work. 

Allegation  11.  SIMA  personnel  who  have  responsibility  for  overseeing  the 
progress  of  the  contracting  effort  have  been  willfully  negligent  in  their 
duties  and  have  conspired  to  exclude  SIMA  programmers  from  the 
contracting  process.  This  negligence  has  resulted  in  wasteful  spending  on 
Government  contracts. 

Audit  Results.  Allegation  11  is  not  substantiated.  Audit  results  did  not  support 
that  SIMA  personnel  were  willfully  negligent.  We  did  identify  one  case,  the 
personal  computer  case  print  application,  in  which  a  SIMA  employee  purposely 
excluded  a  SIMA  programmer  from  the  contract  process.  However,  we  found 
no  evidence  that  the  exclusion  of  the  programmer  represented  conspiracy. 

Allegation  12.  The  contractor  may  have  submitted  false  hourly  billing 
statements  against  ceiling  price  contracts. 

Audit  Results.  Allegation  12  is  not  substantiated.  We  did  not  identify  false 
hourly  billings  being  submitted  by  the  contractor. 

Allegation  13.  The  contractor  was  tasked  to  develop  the  M204  case  print 
program  after  SIMA  employees  had  already  developed  a  M204  case  print 
program,  and  the  SIMA-developed  case  print  program  worked  as  well  as  or 
better  than  the  contractor-developed  case  print  program. 

Audit  Results.  Allegation  13  is  partially  substantiated.  The  contractor  was 
tasked  to  develop  a  M204  case  print  program  after  SIMA  employees  had 


^Privacy  Act  information  deleted. 


38 


Appendix  B.  Summary  of  Allegations  and  Audit  Results 


developed  a  M204  case  print  program  that  printed  with  minor  technical 
deficiencies.  We  did  not  substantiate  that  the  SIMA-developed  M204  case  print 
program  was  better  than  the  contractor-developed  M204  case  print  program. 
See  Appendix  C  for  more  details  on  both  M204  case  print  programs. 
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Comparison  of  Contractor-  and  SEMA-Developed  M204  Case  Print 
Programs.  Our  comparison  test  of  the  contractor-  and  SIMA-developed  M204 
case  print  programs  showed  that  both  programs  were  capable  of  printing  cases. 
We  performed  a  comparison  test  of  both  case  print  programs  to  adequately 
answer  allegation  13.  The  allegation  stated  that  the  contractor  was  tasked  to 
develop  a  M204  case  print  program  previously  developed  by  SIMA  and  that  the 
SIMA-developed  M204  case  print  program  worked  as  well  as  or  better  than  the 
contractor-developed  M204  case  print  program. 

We  conducted  the  comparison  test  at  USASAC-New  Cumberland  in  June  1994 
with  the  assistance  of  an  Inspector  General,  DoD,  software  engineer.  We 
selected  USASAC-New  Cumberland  as  the  site  to  run  the  tests  because  it  was 
the  only  site  at  which  the  SIMA-developed  program  had  been  installed.  To 
perform  the  comparison,  we  statistically  selected  and  printed  54  cases  on  both 
systems  (for  a  total  of  108  printed  cases).  We  requested  that  two  security 
assistance  personnel  responsible  for  writing  cases  at  USASAC-New  Cumberland 
review  both  sets  of  54  cases  and  identify  any  errors  or  differences  with  the 
2  sets  of  cases. 

Our  review  of  the  contractor  version  identified  28  different  types  of  errors  on 
the  54  printed  cases.  A  description  of  the  errors  and  the  number  of  occurrences 
for  each  error  type  are  shown  in  Table  C-l. 

Table  C-l.  Errors  Identified  on  the  Contractor  Version 


Error 


Error  Type 


Number  of 
Occurrences 


1  One  extra  line  space  on  page  3  1 

2  Country  address  should  be  in  three  blocked  lines  7 

3  Incorrect  dollar  amount  on  page  2  2 

4  Missing  data  ("See  note  4")  after  line  item  1  1 

5  Missing  data  ("See  notes  4  and  6")  after  line  item  2  1 

6  Data  printed  on  the  wrong  line  5 

7  Printer  error  when  printing  the  seal  2 

8  Extra  characters  printed  at  the  top  of  the  page  8 

9  Blank  space  as  a  result  of  a  file  change  2 

10  "(T14.H1)"  printed  on  the  wrong  line  14 

1 1  Missing  period  in  note  1  1 

12  Title  line  in  note  7  is  left  justified,  misaligning  the  column  data  1 
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Table  C-l.  Errors  Identified  on  the  Contractor  Version  (cont'd) 


Number  of 

Error  _ Error  Type _  Occurrences 


13  Financial  worksheet  dollar  amounts  should  be  the  same  11 

14  Data  in  note  9  should  not  wrap  1 

15  Data  in  note  2  should  not  wrap  1 

16  Missing  "(  )"  in  column  5  on  page  2  1 

17  Dollar  amounts  in  a  case  notes  schedule  did  not  line  up  14 

1 8  Data  in  note  8  should  not  wrap  1 

19  Wording  different  from  SIMA  version  1 

20  Words  are  wrapping  differently  than  on  SIMA  version  1 

21  Incorrect  word  order  in  note  16  1 

22  Incorrect  word  order  in  note  26  1 

23  Word  with  a  space  in  the  middle  on  page  4  1 

24  Data  in  case  notes  schedule  did  not  line  up  1 

25  Cost  schedule  did  not  line  up  4 

26  Payment  schedule  did  not  line  up  1 

27  Missing  data  and  dollar  amounts  on  page  1  1 

28  Missing  revised  dollar  amounts  on  page  2  1 


Our  review  of  the  SIMA  version  identified  27  different  types  of  errors  on  the 
54  printed  cases.  A  description  of  the  errors  and  the  number  of  occurrences  for 
each  error  type  are  shown  below. 

Table  C-2.  Errors  Identified  on  the  SIMA  Version 


Error 


Error  Type 


Number  of 
Occurrences 


1  Double  line  spaces  between  each  note  49 

2  Missing  financial  worksheet  54 

3  Blank  space  as  a  result  of  a  file  change  2 

4  Missing  dollar  symbol  and  dollar  amounts  4 

5  Incorrect  spacing  between  "U.S.  Government"  1 

6  Missing  line  space  between  note  5  and  note  6  1 

7  Missing  line  space  between  note  12  and  note  13  1 

8  Missing  line  space  between  note  18  and  note  22  1 

9  Missing  last  two  lines  of  data  in  note  17  1 

10  "(T14.H1)"  printed  on  the  wrong  line  6 

1 1  Missing  a  line  space  and  data  before  note  1  1 

12  Dollar  amounts  in  case  notes  schedule  did  not  line  up  14 
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Table  C-2.  Errors  Identified  on  the  SIMA  Version  (cont'd) 


Number  of 


Error 

Error  Tvne 

Occurrences 

13 

Triple  line  spacing  after  note  8 

1 

14 

Extra  line  spacing  on  the  cover  page 

1 

15 

Missing  space  between  words  on  cover  page 

2 

16 

Two  tilde  "~H  marks  at  the  end  of  a  paragraph 

2 

17 

Incorrect  wording  on  page  2 

1 

18 

Amendment  paragraphs  instead  of  modification 

1 

19 

Leading  zeros  on  page  2  should  not  print 

1 

20 

Missing  line  space  before  line  item  1 

8 

21 

Word  with  a  space  in  the  middle  on  page  4 

1 

22 

Data  in  case  notes  schedule  did  not  line  up 

1 

23 

Cost  schedule  did  not  line  up 

3 

24 

Missing  dollar  amount  on  page  2 

1 

25 

Missing  payment  schedule  on  page  2 

2 

26 

Missing  data  and  dollar  amounts  on  page  1 

1 

27 

Missing  revised  dollar  amounts  on  page  2 

1 

Correcting  the  Errors.  We  determined  that  most  errors  could  have  been 
readily  corrected  by  a  skilled  programmer.  In  August  1994,  we  discussed  the 
errors  identified  on  the  test  cases  printed  on  the  two  M204  case  print  programs 
with  both  SIMA  and  the  contractor. 

Correcting  Errors  in  the  Contractor  Version.  Of  the  28  errors 
identified  on  the  contractor  version,  6  (errors  3,  4,  5,  13,  27,  and  28  in 
Table  C-l)  were  considered  to  be  more  important  than  the  other  errors  because 
these  errors  involved  missing  or  incorrect  data.  At  the  time  of  our  review,  four 
errors  (3,  5,  27,  and  28)  had  either  been  fixed  or  were  attributed  to  errors  in  the 
data  base  and,  therefore,  were  not  the  result  of  the  print  program.  We 
determined  that  only  two  of  the  remaining  errors  on  the  contractor's  version 
were  substantial  (errors  4  and  13).  (We  defined  substantial  as  requiring  as 
much  as  1  week  for  correction.)  We  estimated  that  errors  4  and  13  require  up 
to  1  staff-week  of  effort  to  correct.  We  discussed  the  errors  with  the  contractor 
staff  and  they  agreed  with  our  assessment.  According  to  the  contractor, 
one  error  (error  13)  had  been  partially  corrected  during  a  recent  software  release 
and  would  be  fully  corrected  in  the  next  release.  We  identified  the  remaining 
error  (error  4)  to  the  SIMA  SA3  PM. 

Correcting  Errors  in  the  SIMA  Version.  Using  the  same  criteria,  we 
evaluated  errors  generated  by  the  SIMA  version  of  the  M204  case  print 
program.  Of  the  27  errors,  9  (errors  2,  4,  9,  11,  18,  24,  25,  26,  and  27  in 
Table  C-2)  involved  missing  or  incorrect  data.  Of  the  9  errors,  3  (errors  11, 
26,  and  27)  were  attributable  to  errors  in  the  data  base.  Of  the  remaining 
six  errors,  two  (errors  9  and  18)  were  considered  to  involve  simple  fixes,  for 
example,  re-entering  the  data.  We  estimated  that  three  (errors  4,  24,  and  25)  of 
the  other  four  errors  could  be  corrected  within  several  weeks.  We  believe  that 
several  staff-months  of  effort  would  be  needed  to  correct  the  most  significant 
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error,  the  missing  financial  worksheet  (error  2).  We  discussed  our  assessment 
with  the  SIMA  programmer.  The  SIMA  programmer  believed  that  all  errors, 
except  the  missing  financial  worksheet,  could  be  corrected  in  1  to  2  weeks. 

The  SIMA  programmer  also  stated  that  he  was  originally  told  not  to  develop 
M204  case  print  capabilities  for  the  financial  worksheet  because  a  second 
contractor  was  developing  the  financial  worksheet.  Nevertheless,  in 
August  1993,  2  months  before  the  SIMA  work  on  the  M204  case  print  program 
was  terminated,  the  SIMA  programmer  was  tasked  to  develop  the  financial 
worksheet  because  the  contractor  effort  was  unsuccessful.  The  SIMA 
programmer  stated  that  he  requested  guidance  from  the  USASAC  support  office 
but  did  not  receive  guidance  from  USASAC  or  SIMA  before  he  was  removed 
from  the  project. 

Type  of  Printer  Used.  We  performed  our  test  of  the  SIMA-developed  print 
program  on  the  Model  47  printer  because  the  program  was  written  for  that 
printer.  The  Model  47  printer  was  being  replaced  by  the  Model  37  printer  at 
the  major  subordinate  commands;  however,  the  SIMA  programmer  stated  that 
the  Security  Assistance  Functional  Support  Office  would  not  provide  him  access 
to  the  newer  Model  37  printer.  A  review  of  the  source  code  indicated  that  the 
SIMA-developed  print  program  could  be  written  to  accommodate  both  printers. 
However,  we  did  not  test  that  feature. 

Chronology  of  M204  Case  Print  Development.  The  following  provides  a 
chronology  of  events  for  the  M204  case  print  program  development. 

o  In  the  first  quarter  1990,  SIMA  programmers  were  verbally  tasked  to 
develop  the  M204  conversion  with  M204  case  print  capabilities  (referred  to  in 
this  appendix  as  M204  case  print  program). 

o  In  the  third  quarter  1991,  M204  case  print  program  was  installed  at 
US  AS  AC-New  Cumberland  for  prototype  testing,  with  updates  and 
improvements  fielded  in  the  first  quarter  1993. 

o  In  the  third  quarter  1993,  SIMA  programmer  was  tasked  to  develop 
financial  worksheet. 

o  In  the  fourth  quarter  1993,  USASAC  and  SIMA  management  agreed 
to  terminate  work  on  the  SIMA-developed  M204  case  print  program  and 
assigned  the  task  to  the  contractor. 

o  In  the  first  quarter  1994,  the  contractor  version  of  the  M204  case  print 
program  prototype  was  tested  at  the  Army  Armament,  Munitions,  and  Chemical 
Command,  an  AMC  major  subordinate  command. 

o  In  the  second  quarter  1994,  the  contractor  version  of  the  M204  case 
print  program  was  fielded  at  the  four  remaining  AMC  major 
subordinate  commands. 

o  In  the  second  quarter  1994,  the  contractor  version  of  the  M204  case 
print  program  was  fielded  at  US  AS  AC-New  Cumberland. 
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Lack  of  Coordination.  SIMA  management  did  not  allow  SIMA  programmers 
to  coordinate  with  the  USASAC-New  Cumberland  customers  who  were  using 
the  SIMA-developed  M204  case  print  program.  This  case  print  program  was 
initially  fielded  at  USASAC-New  Cumberland  during  the  third  quarter  1991 
and  was  subsequently  updated  during  the  first  quarter  1993.  Following  the 
update,  SIMA  programmers  expected  to  receive  environmental  system  test 
reports  (SIMA  Forms  366a)  or  system  change  requests  from  customers 
reporting  problems  that  were  identified  during  system  use.  Because  SIMA 
programmers  did  not  receive  a  significant  number  of  deficiency  reports,  they 
thought  the  program  was  working  well.  If  SIMA  programmers  had 
communicated  with  the  users  at  USASAC-New  Cumberland,  they  would  have 
found  that  many  users  were  not  using  the  M204  case  print  program  because  of 
errors.  The  users  were  frustrated  with  the  M204  case  print  program  and  most 
users  went  back  to  developing  cases  off-line  using  word  processors. 

We  found  other  indications  that  the  SIMA-developed  M204  case  print  program 
could  have  worked  if  proper  coordination  had  taken  place  between  SIMA  and 
USASAC.  For  example,  one  problem  USASAC-New  Cumberland  users 
experienced  while  printing  cases  was  not  related  to  the  print  program.  The 
printer  used  was  old  and  printed  lines  across  the  cases,  making  the  cases 
unusable.  Further,  one  case  writer  at  USASAC-New  Cumberland  was  willing 
to  work  with  the  print  program  and  the  printer.  This  user  liked  the  SIMA 
M204  case  print  program  and  used  the  program  regularly  until  an  unrelated 
computer  file  added  to  the  USASAC-New  Cumberland  computer  system  caused 
the  SIMA  M204  case  print  program  to  stop  working  around  December  1993. 

SIMA  programmers  were  not  aware  that  the  M204  case  print  program  at 
USASAC-New  Cumberland  stopped  working.  Because  SIMA  programmers 
received  minimal  deficiency  reports,  they  continued  to  believe  the  print  program 
was  working  and  being  used.  If  the  SIMA  programmers  had  communicated 
with  the  users,  they  would  have  found  that  a  file  had  been  added  to  the 
USASAC-New  Cumberland  system  that  caused  the  print  program  to  stop 
printing.  A  SIMA  programmer  identified  this  file  in  June  1994  while  the  audit 
team  was  running  the  comparison  test  of  SIMA-  and  contractor-developed  print 
programs.  Once  the  file  was  deleted  from  the  USASAC-New  Cumberland 
system,  we  were  able  to  print  cases  using  the  SIMA-developed  M204  case 
print  program. 

Conclusion.  We  did  not  substantiate  the  complainant's  allegation  that  the 
SIMA  version  was  better  than  the  contractor  version.  While  both  print 
programs  were  capable  of  printing  cases,  the  contractor's  version  had  errors  that 
could  be  fixed  in  a  more  timely  manner.  The  lack  of  communication  between 
SIMA  and  the  users  at  USASAC-New  Cumberland  caused  a  delay  in  the 
successful  completion  of  the  SIMA-developed  M204  case  print  program.  We 
also  discussed  the  results  of  our  comparison  with  the  SIMA  SA3  PM.  The 
current  SIMA  SA3  PM  was  assigned  to  the  position  in  November  1993,  which 
followed  the  contractor  tasking  to  develop  the  M204  case  print  program.  She 
was  surprised  to  hear  that  the  SIMA  version  of  the  M204  case  print  program 
actually  printed  cases.  She  was  told  that  the  SIMA-developed  M204  case  print 
program  did  not  work. 
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Appendix  D.  Cost  Avoidance  on  Future  Security 
Assistance  Automation,  Army, 

Development  Efforts 


DSAA  should  discontinue  future  funding  of  SA3  development  and  maintenance 
efforts  until  USASAC  identifies  essential  maintenance  in  accordance  with 
current  Joint  Logistics  Systems  Center  initiatives.  Because  SIMA  and  USASAC 
accounting  systems  failed  to  account  for  SA3  development  and  maintenance 
costs,  we  calculated  our  funding  reduction  as  follows. 

Potential  Reduction  in  SA3  Funding 


Fiscal 

SIMA 

USASAC 

Year 

Actual  Funding 

Actual  Funding 

1990 

$800,700 

— 

1991 

917,700 

— 

1992 

903,600 

$2,395,234 

1993 

929,000 

2,460,529 

1994 

919,200 

2,443,778 

Average  funding 

$894,040 

$2,433,180 

85-Percent 

$200,000 

Reduction2 

Reduction1 

Proposed  average  reduction 

$759,934 

$2,233,180 

Total  average  reduction  (SIMA  plus  USASAC) 

$2,993,114 

AMC  major  subordinate  commands 

700.0003 

Total 

$3,693,114 

*We  calculated  that  SA3  requires  3  staff-years  of  effort  at  USASAC  for  systems 
maintenance,  which  equates  to  $200,000  per  year.  Funds  not  shown  for  FYs 
1990  and  1991  because  actual  breakdown  was  not  available;  includes  contractor 
services,  automated  data  processing  equipment,  and  personnel  costs. 

2From  FYs  1990  through  1994,  85  percent  of  SIMA  funding  was  for  SA3 
development.  Therefore,  SIMA  funding  should  be  reduced  by  85  percent. 

3Estimate  based  on  AMC  major  subordinate  commands  supporting 
24  staff-years  of  effort.  We  estimate  that  1  staff-year  of  effort  at  each  of  the 
five  AMC  major  subordinate  commands  and  at  USASAC -New  Cumberland  is 
necessary  for  minimum  SA3  maintenance  cost.  Therefore,  cost  avoidance  of 
$700,000  will  result  when  personnel  billets  for  the  18  staff-years  of  effort  are 
eliminated  as  a  result  of  SA3  budget  reductions. 
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Support  Security  Assistance  at  Major 
Subordinate  Commands 
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PRKrACS 

The  VISION  2000  Security  Assistance  (SA)  Process  Action  Team 
( PAT )  reguastod  that  the  U.  fi.  Army  Materiel  Command  (AMC)  Major 
Subordinate  Command*  (MSC)  Security  Assistance  Management  Director¬ 
ates  r  SAMD)  /  International  logistics  Directorates  (ILD)  (  the 
directorate  )  be  visited  to  review  the  current  state  of  automation 
«nd  to  make  recommendations  to  the  PAT  regarding  incorporation  of 
unique  MSC  requirements  into  the  proposed  VISION  2000  Security 
Assistance  structure.  The  review  was  conducted  at  the  six  MSC'e 
between  12  March  and  20  March  1991  by  Mr.  Robert  M.  Singer,  USASAC 
Director  of  Information  Management  (DOIM)  and  Mr.  Chester  Jay 
Freedcnthal ,  Acting  Product  Manager,  Security  Assistance  Automa¬ 
tion,  Army  (SA3).  Each  review  consisted  of  an  entrance  interview 
with  the  Director,  followed  by  detailed  interviews  and  discussions 
with  the  supervisory  and  functional  personnel  directly  involved  in 
Directorate  automation.  In  some  instances  discussions  with 
representatives  of  the  MSC  DOIM  were  also  held.  The  criteria  on 
which  the  interviews  were  based  consisted  of  1)  Systems  Administra¬ 
tion,  2)  Ha  rdware/Sof tware/Communication,  3)  Training  and  4) 
Standard /Unique  Systems. 

EXECUTIVE  SUMMARY 


The  utilization  of  automation  at  each  MSC  can  be  attributed 
to  a  number  of  related  factors  i  a)  embracing  of  automation  by 
senior  management,  b)  organizational  structure  within  the  director¬ 
ate  to  support  automation/  c)  overall  automation  support  provided 
by  the  MSC  DOIM  and,  d)  assessed  need  for  automation  tools  by  the 
directorate . 

All  of  the  MSC ' s  operate  in  the  Commodity  Command  Standard 
System  (CCSS)  mainframe  and  the  SA3  (Sperry)  minicomputer  environ¬ 
ment,  and  have  adopted  a  microcomputer  (PC)  environment  to  varying 
degrees.  The  organizational  structure  to  support  automation  varies 
considerably  among  the  HSC's,  from  a  low  of  one  at  CECOM  to  eight 
at  AMCCOM.  All  are  organized  to  support  both  the  standard  system 
through  a  Functional  Coordinating  Group  (FCG)  representative ,  and 
office  automation  and  unique  MSC  requirements  utilizing  the  same  or 
different  individuals. 

SA3  Case  Development  and  Case  Management  tools  are  being  used/ 
supplemented  by  locally  developed  tools.  The  automated  SA3 
international  bogistics  Supply  Delivery  Plan  (ILSDP)  has  not  found 
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Subordinate  Commands 


great  acceptance  except  at  AMCCOM,  The  MSC's  were  generally 
satisfied  with  the  support  provided  by  SA3  and  the  SAFSO  office, 
but  felt  that  there  were  still  many  thing®  that  could  improve  the 
SA3  system.  Standard  non-SA3  CCSS  applications  (i.e.,  Cooperative 
Logistics  Supply  Support  Arrangements  (CLSSA),  Concurrent  Spare 
Parts)  are  used  on  an  equal  basis  at  all  MSC'b.  Most  MSC's  voiced 
a  need  for  additional  software  and  Bystems  training,  need  for 
additional  or  replacement  hardware  and  the  willingnea®  to  share 
locally  developed  tools.  While  many  of  the  tools  were  created  to 
satisfy  peculiar  MSC  requirements,  they  essentially  serve  the  same 
purpose®  and  therefore  could  be  standardised# 

MICOH  is  currently  the  only  directorate  that  reimburses  the 
DOIM  for  service®  and  support;  some  of  the  other  MSC's  have 
recently  received  notice  that  they  will  have  to  similarly  reimburse 
in  the  near  future.  This  will  increase  the  budget  requirements  for 
automation  in  the  coming  years.  There  are  many  actions  that  can  be 
taken  in  the  Bhort-term  to  improve  automation,  save  resource®  and 
move  to  a  standardized  environment,  resulting  in  less  turbulence  as 
organizational  changes  occur.  Automation  planning  must  play  a 
significant  roll  in  the  VISION  2000  SA  PAT. 

MSC  REVIEW  AND  ANALYSIS 


AMCCOM 

SUMMARY i  AMCCOM  is  the  moat  progressive  of  the  MSC' a  for 
automation.  Management  support  is  pervasive  and  is  reflected  in  the 
significant  efforts  the  directorate  has  accomplished.  They  have  the 
largest  and  most  diversified  organization  to  support  automation, 
which  provide®  an  important  advantage.  The  working  relationship 
with  the  DOIM  is  excellent,  even  to  having  an  analyst  assigned  to 
the  directorate.  All  systems  administration  and  office  automation 
support  la  accomplished  within  the  directorate.  SA3  applications 
are  utilized  to  the  maximum,  including  structured  language  querying 
of  databases.  They  have  created  an  automated  process  to  e-mail  a 
response  to  the  maker  of  the  query,  which  could  be  useful  to  the 
other  MSC's.  In  addition,  they  have  taken  the  lead  in  Unix  applica¬ 
tion  development  with  Oracle  databases  developed  for  Special 
Defense  Acquisition  Pund  and  ILSDP  management.  They  are  the  only 
MSC  to  fully  utilize  the  SA3  automated  ILSDP.  Every  individual  ha® 
a  terminal,  with  PC  '  a  being  used  by  management  and  for  special 
programs?  access  is  through  the  AMCCOM  Local  Area  Network  (LAN). 
E-mail  Is  used  extensively.  The  standard  suite  of  PC  and  Unix 
software  is  used  and  group  and  individualized  training  la  conduct¬ 
ed,  The  pat  process  has  been  used  to  improve  SA3  and  local 
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automation.  Many  of  the  AMCCCM  developed  automation  toole  could  be 
utilized  to  significant  advantage  by  the  other  MSC's. 

1.  SYSTEMS  ADMINISTRATION! 

a.  Large  dedicated  staff. 

b.  Self  supporting  and  well  trained. 

c.  Works  well  with  DOIM. 

d.  Full  utilization  of  e-mail 

2 .  HARDWAKE/SOFTWARE/COMMUNICATIONi 

a.  All  hardware/software  meet  requirements  to  support  current 
mi a »ion/f unction . 

b.  Received  a  special  requirement  from  mG  Lightner  for  a 
publishing  system  to  produce  a  “Training  Devices  Brochure "  for  PM 
trade.  Requires  purchasing  Desktop  Publishing  hardware/software. 

c.  Long  lead  times  (9-12  months)  to  procure  through  DOIM. 

3.  TRAINING! 

a.  Need  Model  204  training. 

b.  Need  SA3  training  package. 

4.  STANDARD  SYSTEMS I 

a.  Fully  utilize  SA3  Case  Development  and  Management  System; 
using  SA3  Automated  ILSDP;  transmits  cases  electronically. 

b.  Have  developed  numerous  UNIX-based  systems  that  can  be 
ported  to  other  commands  (Scorecard ,  Case  Tracking,  Automated  S2K 
Query,  etc . ) . 

c.  Willing  to  Bhare  systems. 

d.  willing  to  provide  on-site  assistance  to  other  MSC's. 

e.  Has  had  problems  with  Oracle  download.  Also  experienced 
excessive  run  time  for  SA3  applications. 
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AVSCOM 

SUMMARY!  AVSCOM  haa  a  strong  staff  of  four  to  support 
automation,  but  management  support  is  less  evident  than  at  other 
MSC's,  They  appear  to  have  only  a  pedestrian  relationship  with  the 
DOIM.  While  SA3  Is  used  to  prepare  most  cases,  electronic  transmis¬ 
sion  la  not  utilized  because  of  communication  problems  and  SA3  case 
design  differs  from  AVSCOM  case  design  and  many  manual  changes  are 
made  to  cases,  bocal  automation  tools  have  been  developed,  the 
moat  significant  being  the  Case  Management  Plan.  This  program 
closely  aligns  with  the  Case  Evaluation  Profile  and  should  be 
looked  at  closely  for  its  potential  as  a  standard  tool.  Almost  all 
individuals  have  terminals  with  access  through  the  AVSCOM  LAN.  The 
standard  euite  of  PC  and  Unix  software  is  used.  There  is  limited 
use  of  e-mail.  The  directorate  is  acquiring  Bernoulli  Boxes  for 
classified  processing,  System  administration  is  provided  by  the 
DOIM,  with  initial  help  desk  support  provided  by  Directorate 
personnel , 

1.  SYSTEMS  ADMINISTRATION! 

a.  Systems  administration  are  non-ADP  personnel. 

b.  Lack  training  required  to  perform  function  of  system 
administrator , 

c.  Receive  system  administration  support  from  DOIM. 

d.  E-mail  not  integrated  into  systems. 

2  .  HARDWARE/SOFTWARE/COKMUNICATIONS I 

a.  Problem  with  KNET, 

b.  Do  not  use  DDN  to  send  cases  due  to  no  connection  to  DOIM 

DDN . 

c.  Long  lead  time  (9  months)  to  procure  through  DOIM. 

d.  Have  a  need  for  additional  equipment /software  due  to 
expanding  staff. 

3.  TRAINING i 

a.  Need  SA3  training  package. 
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b.  Need  system  administration  and  Model  204  training, 

c.  Would  like  MC"  Programming  training. 

4.  STANDARD  SYSTEMS  I  , 

a.  Use  SA3  to  develop  case/  but  make  manual  changes;  do  not 
submit  electronically.  Does  not  use  Oracle  download. 

b.  Strong  support  and  utilization  of  non-SA3  CCSS  applica¬ 
tions  . 

c.  Have  developed  systems  that  can  be  ported  to  other  MSC's. 

d.  Does  not  use  Oracle  download. 


CECOH 

SUMMARY*  CECOM  has  only  one  individual  working  automation.  All 
systems  administration  is  performed  by  the  DOIM.  The  DOIM  maintains 
complete  control  over  the  minicomputers  and  affords  very  limited 
access  to  the  directorate/  the  relationship  is  good  but  tight 
controls  hinder  flexibility.  E-mail  is  limited  and  is  not  located 
on  the  directorate  Sperry.  The  one  individual  provides  functional 
support  to  the  users  for  both  CCSS  standard  systems  and  local 
automation.  Management  support  of  automation  is  evident  but  not 
exploited.  Sa3  is  used  for  case  development  and  management  and 
caBes  are  electronically  transmitted.  Terminals  are  available  to 
most  individuals  and  access  is  through  the  CECOH  LAN.  The  standard 
suite  of  PC  and  Unix  software  is  used.  Limited  local  user  tools 
have  been  developed  using  PC  based  software.  Some  may  be  usable  by 
other  MSC's,  but  they  do  not  appear  to  be  as  strong  as  Unix  based 
tools. 

1.  SYSTEMS  ADMINISTRATION! 

a.  System  administration  accomplished  by  CECOM  DOIM. 

b.  ILD  automation  functional  has  no  system  access  to  the  SPERRY 
or  use  of  ORACLE;  e-mail  (MMDFII)  not  on  IL  SPERRY. 

c.  Small  staff  (one  non-ADP  person)  provides  system  adminis¬ 
tration  assistance. 

d.  Needs  training  on  system  administration. 
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2.  HARDWARE/ SOFTWARE/ COMMUNICATION I 

a.  Long  lead  time  (9-12  months)  to  buy  through  CBCOK  DOIM. 

b.  Need  additional  hardware/software. 

c.  System  acceBfl  okay. 

d.  No  problem  with  KNET. 

3.  TRAININGi 

a.  Need  system  administration  training. 

b .  Need  SA3  training. 

4.  STANDARD  SYSTEMS* 

a.  Strong  user  of  CCS S  Standard  Systems;  SA3  being  used  for 
most  cases;  electronic  transmission  also  used.  Limited  use  of  case 
management  tools.  Cannot  use  Oracle  download, 

b.  Extensive  use  of  PC  based  user  tools. 

c.  Would  like  to  review  Unix  systems  developed  by  the  other 
commands . 


SUMMARY!  MICOM  has  a  strong  staff  of  seven  to  support 
automation  and  has  been  automated  the  longest  of  any  MSC.  They  have 
an  excellent  relationship  with  the  DOIM  and  work  very  closely 
together.  They  are  currently  the  only  Directorate  that  reimburses 
the  DOIM  for  its  services.  Management  support  haa  not  been  as 
strong  as  it  currently  is,  so  there  has  been  a  relatively  slow 
acceptance  of  the  use  of  SAB  tools  in  favor  of  locally  developed 
tools.  All  systems  administration  functions  are  performed  by  the 
DOIM,  but  the  directorate  has  open  access  to  databases  and 
software.  MICOM  is  a  very  strong  user  of  CCSS  Standard  Systems, 
particularly  CLSSA.  All  users  have  access  through  multiplexers  to 
the  extensive  MICOM  computer  network  (largest  of  the  MSC's,  over 
7000  users).  There  is  extensive  use  of  e-mail.  The  standard  suite 
of  pc  and  Unix  software  is  used,  except  for  the  use  of  MICOM 
standard  database  management  systems  on  the  Sperry  computers  in 
lieu  of  Oracle.  There  is  also  a  MICOM  executive  network  of 
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Macintosh  PC's,  the  directorate  haa  ordered  some  of  the  machines  to 
comply  with  the  standard,  They  have  recently  started  using  SA3  to 
prepare  cases  and  will  begin  transmitting  electronically.  There  are 
a  significant  number  of  local  tools  developed  in  both  the  Unix  and 
mainframe  environment  that  should  be  reviewed  for  standardization. 

1.  SYSTEM  ADMINISTRATION * 

a.  Y<’ell  staffed  to  provide  system  administration  functions. 

b.  Receives  excellent  support  from  MICOM  DQIM. 

2  .  HARDWARE/ SOFTWARE /COMMUNICATIONS  t 

a.  Have  had  communications  problems  accessing  Case  Closeout 
Program  at  NCAD. 

b.  Takes  €  to  9  months  to  procure  requirements  through  >.ICOM 
DOIM, 

c.  Have  a  need  for  additional  hardware/software. 

d.  Reimburses  DOIM  for  services/support  provided. 

3.  TRAINING* 

a.  Need  SA3  training  package. 

b.  Need  Model  204  training. 

4,  STANDARD  SYSTEMt 

a.  Very  strong  user  of  CCSS  Standard  Systems.  Have  recently 
started  using  SA3  for  case  development.  Many  local  tools  developed 
in  the  mainframe  and  Unix  environment. 

b.  Have  developed  systems  that  could  be  ported  to  other  NSC's. 


TACOM 

SUMMARY t  A  small  staff  of  two  supports  the  entire  Directorate 
automation  program.  The  SA3  minicomputer  is  located  in  the 
directorate  and  system  administration,  database  administration, 
office  automation,  help  desk  support  and  CCSS  standard  Systems 
management  are  all  the  responsibility  of  the  small  staff.  TACOM  is 

8 


52 


Appendix  E.  Assessment  of  Automation  to  Support  Security  Assistance  at  Major 

Subordinate  Commands 


the  only  MSC  where  the  Sperry  computers  ore  a till  located  within 
the  directorate,  Management  support  of  automation  is  apparent/  but 
could  be  stronger.  There  is  a  weak  relationship  with  the  DOIM, 
resulting  in  leas  than  acceptable  support  in  moat  cases.  The 
standard  suite  of  PC  and  Unix  software  is  used.  Access  to  the 
system  la  through  the  TACOM  LAN.  There  is  a  need  for  additional 
equipment  and  terminals,  SA3  is  used  for  some  case  development  and 
they  have  been  transmitting  cases  electronically,  although  with 
some  communications  difficulty.  They  have  a  variety  of  old 
equipment,  including  an  Intel  310,  which  needs  to  be  replaced.  Moat 
of  the  unique  automation  tools  are  on  the  Intel  and  could  be  moved 
to  the  Sperry  with  some  programming  and  systems  support.  E-mail  is 
not  used  extensively.  TACOM  was  openly  receptive  to  any  assistance 
that  could  be  provided  in  improving  their  automation  program.  Even 
with  weaknesses,  they  have  made  significant  strides  in  a  short 
period  of  time. 

1.  SYSTEMS  ADMINISTRATION! 

a.  Small  non-ADP  staff  supports  the  SPERRY  Computer  Systems. 

b.  Need  more  emphasis  on  having  personnel  dedicated  to  systems 
administration  on  SPERRY  and  PC'S. 

c.  Lack  UNIX  SyBtem  administration  training. 

d.  Need  assistance  in  systems  conf iguratlon/management . 

e.  Receive  little  support  from  DOIM. 

2 ,  HARDWARE/SOFTWARE/COMMUNICATIONS  I 

a.  Lacking  PC/terminals/softwars  and  printers. 

b.  Still  using  INTEL  310  for  processing. 

c.  Need  to  convert  from  INTEL  310  to  SPERRY  ORACLE. 

d.  No  more  porta  available  on  SPERRY. 

e.  Having  problem  with  KNET, 

f.  No  DDN  connection  exists  between  SPERRY  and  mainframe. 

g.  Long  lead  times  to  procure  hardware/cof twar ©/communication 
requirement  through  DOIM. 
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3.  TRAINING; 

a.  Lack  system  administration  and  PC  hardware/software 
training. 

b.  Need  UNIX/ORACLE  DBMS  and  Modal  204  training. 

c.  Need  a  list  of  all  system  and  program  documentation . 

4.  STANDARD  SYSTEMS  * 

a.  Uses  CCSS  Standard  systems  extensively.  Expanding  use  of 
SA3  for  case  development;  has  transmitted  cases  electronically. 

b.  Eager  to  obtain  any  systems  that  are  available  from  other 
MSC's. 

c.  Have  little  to  offer  other  MSC's  but  willing  to  share 
ideas . 

TROSCON 

SUMMARY*  Though  it  is  the  smallest  Directorate  (really  a 
Division  in  the  Materiel  Management  Directorate),  TROSC0M  has  a 
strong  staff  of  two  supporting  automation.  There  is  a  very  good 
working  relationship  with  the  DOIH.  Management  support  is  signifi¬ 
cant  and  Is  looking  for  more  standardisation.  There  was  particular 
frustration  over  the  automated  SA3  ILSDP.  Office  automation  is 
strong  and  some  specialized  tools  have  been  developed,  but  they 
could  benefit  from  standardized  case  management  tools.  Access  is 
provided  through  the  AV5COM  LAN .  All  individuals  have  terminals  and 
PC'®  are  being  used  by  management.  The  standard  suite  of  PC  and 
Unix  software  is  used.  Most  cases  are  prepared  using  SA3  and  are 
transmitted  electronically. 

1.  SYSTEMS  ADMINISTRATION! 

a.  Staff  small  but  performs  all  required  administration. 

2 .  HARDWARE/SOFTWARE/COMMUNICATION! 
a.  Connected  to  DDN. 
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b.  Have  a  problem  with  KNET. 

c.  Need  additional  equipment/Bof tware . 

3  «  TRAINING I 

a.  Need  Model  204  training. 

4.  STANDARD  SYSTEMS! 

a.  Uses  CCSS  Standard  Systems;  transmits  eases  electronically. 
Would  like  to  use  automated  ILSDP,  but  feels  it  has  too  many 
problems . 

b.  Would  like  to  share  in  systems  developed  by  other  MSC's, 


AUTOMATION  ASSESSMENT  AND  RECOMMENDATIONS 


1.  ASSESSMENT/SITUATION  -  Centralize  Automation  Procurement! 

a.  All  ILD/SAMD  indicated  they  go  through  long  lead  times  to 
procure  hardware/sof tware  after  the  CAPR  is  submitted  to  the  DOIM. 
Thio  causes  a  problem  In  providing  real  time  support  to  the  staff 
and  a  delay  in  supporting  requirements. 

-  RECOMMENDATIONS! 

a.  All  MSC 'a  submit  their  hardware/software  requirements  to 
AMSAC-IM  for  central  procurement  and  distribution, 

-  BENEFITS: 

a.  Will  provide  central  control /management  of  all  MSC 
procurement  using  SA3  funds, 

b.  Allow  for  USASAC  to  establish  standards  with  all  MSC's, 

c.  Reduce  long  lead  time  (9-12  months)  to  a  time  of  1-3  months 
after  receipt  of  MSC  CAPR. 


2.  ASSESSMENT/ SITUATION  -  Select  and  Proliferate  Standard  Systems! 


11 
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a.  Various  security  assistance  sub-systems  have  been  developed 
on  both  the  SPERRY  and  the  82X  System,  All  ILD/SAMD  cited  a  need 
to  review  these  systems/  pick  "best-of -breed"  and  proliferate  to 
all  MSC ' 0  thus  establishing  standards, 

-  RECOMMENDATION  I 

a.  Obtain  detailed  documentation  on  each  system  and  form  a  PAT 
to  review  and  select  systems  for  proliferation  plus  determine  the 
MSC  that  will  be  the  proponent  for  system  maintenance  and  distribu¬ 
tion,  USASAC  should  be  included  on  the  PAT  to  insure  system 
interface  requirements  are  considered. 

-  BENEFITS! 

a.  Allows  for  those  KSC's  with  lees  resources  to  obtain  an 
already  developed /documented  system. 

b.  provides  for  standard  systems  within  Security  Assistance. 

c.  Ensures  an  easier  transition  to  Vision  2000. 

d.  Enables  MSC' a  to  share  information  and  resources. 

e.  Provide*  for  increased  accuracy  of  data  in  disbursed 
databases . 

3,  ASSESSMENT/ SITUATION  -  Improve  Comrouni  cat  lone  Protocols* 

a.  The  communications  protocol  between  the  commands  and  USASAC 
(ALEX  and  NCAD)  is  not  compatible  and  creates  a  hardship  in  trying 
to  query  for  information. 

-  RECOMMENDATION! 

a.  AMS AC - I M  send  a  communications  specialist  to  each  command 
to  review,  standardize  and  test  communications  protocol  between 
each  ILD/SAMD  and  USASAC  * 

-  BENEFITS! 

&.  Provide  communication  standards. 

b.  Allow  for  easy  access  to  USASAC  databases, 

12 
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c«  provide  means  for  exchange  of  data. 


4,  ASSBSSXKNT/SITUATION  -  Establish  Standard  Support  Structure* 

a.  The  system  administration  function  on  the  SPERRY  System* 
are  performed  by  various  type  non-ADP  personnel  with  each  IL/SAMD 
staffing  varying  between  on©  person  at  CECOK  to  ©ight  persons  at 
AHCCOM.  Personnel  are,  in  moat  cases,  not  trained  in  all  aspects 
of  systems  administration. 

-  RECOMMENDATION i 

a .  Develop  a  proposed  manning  structure  for  system  administra¬ 
tion  at  each  IL/Samd. 

b.  Develop  a  training  matrix  and  plan  for  system  administra¬ 
tion  personnel. 

c.  Organize  a  system  administration  user  group  by  having 
representatives  from  each  IL/SAMD  meet  annually. 

d.  IM  send  a  systems  administrator  (along  with  a  communica¬ 
tions  specialist)  to  visit  and  work  with  all  IL/SAMD  systems 
administrators  in  solving  problems,  sharing  ideas  and  assisting  in 
developing  standards. 


5.  ASSBSSXKNT/SITUATION  -  Review  ILSDP  Process  and  Automated 
ILSDPi 

a.  The  SA3  automated  ILSDP  has  been  fielded  for  two  years  and 
modified  several  times.  It  is  based  on  the  current  ILSDP  prepared 
using  word  processing.  The  automated  ILSDP,  for  a  number  of 
reasons,  has  only  been  accepted  by  AMCCOM.  Management  of  the  ILSDP 
has  recently  transferred  from  USASAC  Alexandria  to  USASAC  New 
Cumberland.  A  positive  step  needs  to  be  taXen  to  review  the  need* 
of  USASAC  relative  to  ILSDP  type  data  and  reporting  requirements  of 
the  ILSDP. 

RECOMMENDATION I 

a.  Establish  a  PAT  to  review  the  ILSDP  process,  requirements 
for  procurement  data  and  the  ILSDP  report. 
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b.  Exclude  the  current  automated  ILSDP  from  Model  204 
development  pending  results  of  the  PAT* 

c.  Encourage  the  MSC's  to  utilize  the  automated  ILSDP  to  the 
maximum  extent  possible. 

d.  Continue  with  piano  to  transfer  the  MSC  ILSDP  files 
electronically  to  USASAC. 

-  BENEFITS l 

a.  Final  decision  on  need  for  an  ILSDP  or  ILSDP  type  report. 

b.  Improve  accuracy  of  data  acroee  databases, 

6.  ASSeSSMBOT/SITUATION  -  Ensure  SA3  Functionality! 

a.  There  are  many  email  areas  where  policy  fy®11®” 

can* bilitv  of  SA3  differ  to  the  extent  that  some  MSC  a  find  SA3 
unStaMe  for  eome  ca.e..  Moat  of  theae  ereaa  are  the  legacy  of  pre- 
SA3  caae  preparation,  when  word  processing  made  changes  eaey  and 
each  MSC  had  their  own  standards.  Before  full  caM«calon  of SA3  t 
Model  204  theses  areas  need  to  be  resolved.  Additionally,  many  of 
the  MSC 'a  are  experiencing  excessive  run  times  for  SA3  applies- 
tiona.  In  some  instances,  runs  were  aborted  because  of  the  runtime 
involved. 

RECOMMENDATIONS i 

a.  Issue  a  data  call  for  all  MSC'e  to  identify  thoae  areas 
where  they  feel  there  ia  a  contention  between  SA3  and  usasac  or 
local  policy  mandating  how  cases  be  prepared.  Request  that  SIMA 
look  at  the  excessive  runtimes  being  experienced  by  the  MSC  a.  In 
other  words ,  "What’s  wrong  with  6A3  ?**. 

b.  Convene  a  USASAC  management  level  panel  to  review  the  data 
call  and  SIMA  study  of  runtime  and  direct  policy  and  system  changes 
as  appropriate  to  resolve  the  differences. 

c.  Implement  changes  in  the  existing  SA3  system  that  are 
determined  to  be  top  priority.  Insure  that  all  changes  are 
incorporated  into  the  Model  204  system  before  it  is  proliferated. 
Take  required  action  to  optimize  the  runtime  of  SA3  applications. 
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Management  Comments  on  Satisfying  Mission  and  User 
Requirements.  The  Army  stated  that  the  SA3  information  system  is  fully 
operational  and  that  SA3  fully  satisfies  mission  and  user  requirements. 
According  to  the  Army,  cases  have  been  prepared  in  SA3  since  1988  and 
management  modules  have  been  in  use  since  1987.  A  data  call  to  SA3  users  in 
October  1994  demonstrates  mission  and  user  satisfaction.  (See  Part  IV, 
pages  80-97  for  results  of  the  data  call.) 

Audit  Response.  We  maintain  our  position  that  SA3  does  not  fully  satisfy 
mission  and  user  needs.  We  agree  that  SA3  may  have  been  used  to  prepare  a 
few  cases  as  early  as  1988;  however,  we  concentrated  our  review  on  how  much 
SA3  was  used  and  what  it  was  used  for.  SA3  was  not  consistently  used  for  case 
preparation  (referred  to  in  the  finding  as  case  development)  until  M204  was 
fielded  in  April  1994.  We  expected  SA3  usage  for  case  preparation  to  increase, 
stating  in  our  draft  report  that,  according  to  users,  the  M204  data  base 
conversion  made  case  development  easier.  However,  because  the 
M204  conversion  was  only  recently  fielded  at  the  time  of  our  audit,  we  did  not 
evaluate  how  much  SA3  was  used  after  the  conversion. 

Although  timely  and  accurate  case  preparation  is  crucial  to  establishing  an 
FMS  case,  the  actual  preparation  of  the  case  represents  only  a  small  part  in  the 
overall  life  of  an  FMS  case.  Based  on  our  discussions  with  case  managers, 
managing  a  case  once  it  has  been  accepted  by  a  foreign  country  occupies  the 
majority  of  a  case  manager's  time.  The  Army  expected  SA3  to  provide 
significant  assistance  with  case  management  through  the  International  Logistics 
Supply  Delivery  Plan.  As  stated  in  the  finding,  that  application  was  seldom 
used  and  was  deleted  in  1991.  The  Army  maintains  that  another  application, 
the  Security  Assistance  Management  Acquisition  Program,  now  provides  case 
managers  with  a  system  of  monitoring  case  status,  a  user  requirement  that  was 
to  be  satisfied  through  the  International  Logistics  Supply  Delivery  Plan. 
However,  case  managers  stated  that,  although  the  Security  Assistance 
Management  Acquisition  Program  was  available  on  the  SA3  menu,  case 
managers  had  not  received  training  on  the  application  and  were  not  confident 
that  the  application  would  work.  Results  of  the  SA3  data  call  (Part  IV, 
page  88)  confirmed  that,  as  of  December  1994,  the  Security  Assistance 
Management  Acquisition  Program  still  was  not  being  used. 

Management  Comments  on  SA3  Improvements  Resulting  From  Data  Base 
Conversion  to  M204.  The  Army  stated  that  initially  SA3  was  to  be  primarily  a 
standardized  and  centralized  case  preparation  system  that  provided  case 
management  tools.  With  the  availability  of  M204,  the  Army  could  use  the  data 
in  SA3  for  other  applications.  The  Army  further  stated  that  the  improvements 
made  to  SA3  since  FY  1990  should  not  be  linked  to  SA3  for  life  cycle- 
management  purposes.  Specialized  applications  such  as  the  Security  Assistance 
Management  Acquisition  Program  resulted  in  improvements  to  SA3  that  were 
possible  because  of  the  outgrowth  of  M204  advances. 
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Audit  Response.  We  disagree  that  initially  SA3  was  to  be  primarily  a  case 
preparation  system.  As  stated  in  several  SA3  planning  documents  beginning  in 
FY  1983,  a  primary  objective  of  SA3  was  to  "provide  the  required  databases, 
processes,  and  capabilities  necessary  for  total  management  (emphasis  added)  of 
the  Security  Assistance  Program,  during  routine  as  well  as  emergency,  crisis, 
and  mobilization  situations."  We  do  not  believe  this  objective  in  any  way 
emphasizes  case  preparation. 

We  also  disagree  with  the  Army's  statement  that  improvements  made  to  SA3 
since  FY  1990  should  not  be  linked  to  SA3  for  life-cycle  management  purposes. 
Life-cycle  management  does  not  end,  as  Army  comments  implied,  when  an 
information  system  is  fielded.  According  to  Army  life-cycle  guidance,  the 
SA3  PM  was  required  to  obtain  milestone  approval  authority  from  the  Assistant 
Secretary  of  the  Army  (Installations,  Logistics,  and  Environment)  to  develop  the 
M204  conversion,  a  major  modification  to  the  existing  SA3  information  system. 
That  approval  was  to  include  an  update  of  all  previously  approved  baseline  cost, 
schedule,  and  program  requirements.  The  updated  information  could  assist 
management  in  evaluating  whether  the  benefits  to  be  received  from  SA3 
warranted  additional  costs,  which  significantly  exceeded  original  cost  estimates. 
For  cost  information  to  be  useful,  the  actual  investment  cost  of  SA3  should 
include  the  development  and  hardware  costs  necessary  for  the  system  to  satisfy 
approved  basic  mission  and  user  requirements.  The  M204  conversion  should  be 
included  as  an  actual  investment  cost,  because  without  the  M204  conversion, 
SA3  did  not  satisfy  even  the  most  basic  requirement  of  case  preparation. 

Management  Comments  on  the  $46  Million  Cost  Incurred  on  SA3.  The 
Army  stated  that  $46  million  for  SA3  development  was  overstated.  The  Army 
maintains  that  SA3  system  development  was  completed  by  FY  1990,  the  system 
was  fielded,  and  the  project  manager  position  was  abolished  in  February  1991. 
The  Army  further  stated  that  the  audit  report  was  incorrect  in  its  assumption  that 
SA3  remained  in  design  and  development  status  through  FY  1994.  By  failing  to 
recognize  that  SA3  had  been  fielded  for  3  years,  the  report  understated  the 
capabilities  of  SA3. 

Audit  Response.  The  report  did  not  state  that  the  Army  spent  $46  million  for 
SA3  development.  Although  the  Army  considered  SA3  a  fielded  system  in 
FY  1990,  the  Army  continued  to  fund  44  work  years  of  effort  for  SA3  through 
FY  1994  (Part  II,  page  27).  The  Army  was  unable  to  provide  supporting 
documentation  of  a  breakdown  on  SA3  development  and  enhancement  cost 
versus  maintenance  cost.  The  SA3  PM  reported  $34.9  million  spent  on  SA3 
through  FY  1990,  the  year  the  Army  considered  SA3  to  be  a  fielded  system.  In 
addition,  the  $34.9  million  spent  as  of  FY  1990  did  not  include,  at  a  minimum, 
the  cost  to  develop  the  M204  conversion  ($3.4  million)  and  the  Security 
Assistance  Management  Acquisition  Program  ($651,000).  We  revised  the 
report  to  clarify  our  understanding  of  what  costs  are  included  in  the  $46  million. 
(See  Part  II,  page  12.) 

The  year  SA3  was  fielded  has  little  to  do  with  the  capabilities  of  SA3.  A 
fielded  information  system  has  limited  value  if  it  is  seldom  used.  Even  if  we 
assume  that  SA3  is  currently  used  to  prepare  100  percent  of  the  Amy's 
FMS  cases,  SA3  did  not  become  an  effective  tool  for  case  preparation  until  the 
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M204  conversion  was  fielded  in  April  1994.  The  results  of  the  SA3  data  call  as 
of  December  1994  (Part  IV,  pages  80-97)  confirmed  that,  as  far  as  users  are 
concerned,  SA3  capabilities  beyond  case  preparation  remain  limited. 

Management  Comments  on  Controls  Bypassed  on  System  Change.  The 
Army  stated  that  it  could  not  explain  why  established  internal  controls  on  system 
changes  were  bypassed  when  developing  the  M204  conversion.  The  Army 
justified  its  failure  to  obtain  an  approved  system  change  request  by  stating  that 
M204  was  a  legitimate  project  and  that  the  conversion  of  SA3  from  an  S2K  data 
base  to  an  M204  data  base  was  a  needed  change.  The  Army  disagreed  that 
SIMA  inappropriately  used  internal  projects  on  tasks  other  than  the 
M204  conversion. 

Audit  Response.  The  Army's  attempt  to  justify  why  established  internal 
controls  on  system  changes  were  bypassed  when  developing  the  M204 
conversion  indicates  that  SIMA  does  not  understand  the  importance  of  internal 
controls.  We  agree  that  a  change  from  the  initial  S2K  data  base  was  needed  for 
SA3  to  be  used.  However,  the  Army  has  no  way  to  determine  whether  the 
conversion  to  M204  was  the  change  that  was  needed  because  the  Army  did  not 
evaluate  alternatives.  The  established  controls  over  system  changes  provided 
for  the  evaluation  of  cost-effective  alternatives. 

Based  on  the  Army  comments  and  the  insignificance  of  the  other  internal 
projects  in  the  draft  report  we  identified  as  inappropriate,  we  deleted  from  the 
final  report  our  discussion  on  improper  use  of  other  internal  projects.  The 
Army's  planned  action  to  establish  procedures  on  the  use  of  internal  projects  is 
fully  responsive  to  our  concerns  regarding  SIMA's  use  of  internal  projects. 

Army  Comments  on  Capabilities  of  Contractor-Developed  Versus  SIMA- 
Developed  Case  Print  Process.  The  Army  did  not  agree  with  our  conclusion 
that  both  the  SIMA  and  contractor  M204  case  print  programs  were  capable  of 
printing  cases,  with  minor  deficiencies,  stating  that  the  report  failed  to  mention 
that  the  case  print  process  the  SIMA  programmer  developed  did  not  work  with 
the  Model  37  laser  printer.  The  Army  considered  this  deficiency  to  be  critical. 
The  Army  also  stated  that  the  tests  run  at  USASAC-New  Cumberland  were  not 
representative  of  the  cases  that  are  run  at  the  major  subordinate  commands. 
Therefore,  the  report  was  wrong  to  conclude  that  the  SIMA-developed  case 
print  process  had  only  minor  deficiencies. 

Audit  Response.  We  performed  our  test  of  the  SIMA-developed  print  program 
on  the  Model  47  printer  because  the  program  was  written  for  that  printer.  The 
SIMA  programmer  stated  that  the  Security  Assistance  Functional  Support  Office 
would  not  provide  him  access  to  the  newer  Model  37  printer.  A  review  of  the 
source  code  indicated  that  the  SIMA-developed  print  program  could  be  written 
to  accommodate  both  printers.  We  revised  Appendix  C  to  include  a  discussion 
of  the  printer  used  during  the  test.  As  stated  in  Appendix  C,  we  selected 
USASAC-New  Cumberland  as  the  site  to  ran  the  tests  because  it  was  the  only 
site  at  which  the  SIMA-developed  program  had  been  installed.  We  never  stated 
or  implied  that  the  cases  ran  at  USASAC-New  Cumberland  are  representative  of 
the  cases  that  are  run  at  the  major  subordinate  commands. 
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Recommendation 

Reference  Description  of  Benefit 


Amount  and 
Type  of  Benefit 


l.a.  Economy  and  Efficiency.  Requires 

compliance  with  existing 
DoD  guidance  and  DoD  CIM 
initiatives  requiring  no 
further  DoD  information 
system  development. 


l.b.  Economy  and  Efficiency.  Requires 

DSAA  to  fund  only  essential 
maintenance  on  SA3. 

l.c.  Economy  and  Efficiency.  Requires 

SA3  funds  to  be  spent  on  most 
cost-effective  alternative. 

1. d.  Internal  Controls.  Verifies  that  the 

Army  performs  life-cycle 
management  and  accounts  for  funds 
expended  for  SA3. 

2.  a.  Internal  Controls.  Establishes 

procedures  to  approve  SA3  system 
changes  and  to  track,  update,  and 
report  SA3  costs  as  required. 

2.b.  Internal  Controls  and  Compliance 

with  Laws  or  Regulations.  Verifies 
compliance  with  existing  life-cycle 
management  regulations. 

2.c.  Economy  and  Efficiency.  Identifies 

all  SA3  costs,  including 
personnel  requirements. 

See  footnotes  at  end  of  appendix. 


Avoidance  of  as  much 
as  $3.7  million  of 
SA3  development 
costs  annually  would 
allow  appropriation 
97XX  funds  to  be  put 
to  better  use  on 
standard  FMS 
information  system. 1 

Undeterminable. 1 


Undeterminable. 1 


Nonmonetary. 


Nonmonetary. 


Nonmonetary. 


Nonmonetary. 
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Appendix  G.  Summary  of  Potential  Benefits  Resulting  From  Audit 


Recommendation 

Reference  Description  of  Benefit 


Amount  and 
Type  of  Benefit 


2.d. 

Economy  and  Efficiency.  Provides 
DSAA  with  accurate  information  on 
status  of  SA3  to  decide  future  of 

FMS  automation. 

Undeterminable. 

2.e. 

Economy  and  Efficiency.  Avoids 
unnecessary  SA3  support  costs  that 
occurred  through  duplication 
of  effort. 

Undeterminable, 

3. a. 

Internal  Controls.  Requires  SA3  to 
be  properly  documented. 

Nonmonetary. 

3.b. 

Internal  Controls.  Limits  use  of 
internal  projects  to  administrative 
support  functions. 

Nonmonetary. 

3.c. 

Internal  Controls.  Requires 

SA3  system  changes  to  be  made 
with  an  approved  change  request. 

Nonmonetary. 

3.d. 

Internal  Controls.  Verifies 
contractor  work  orders  include 
required  supporting  documents. 

Nonmonetary. 

3.e. 

Internal  Controls.  Requires 
preparation  of  design  and  test  plans 
before  performing  additional 

SA3  development,  maintenance, 
or  enhancement. 

Nonmonetary. 

'Actual  monetary  benefits  will  be  determined  when  USASAC  identifies  and 
justifies  essential  maintenance  costs  for  SA3. 

2Monetary  benefits  were  undeterminable  because  the  resources  of  the  Security 
Assistance  Functional  Support  Office  could  be  utilized  elsewhere. 
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Appendix  H.  Organizations  Visited  or  Contacted 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition  and  Technology,  Washington,  DC 
Assistant  Deputy  Under  Secretary  of  Defense  (Logistics  Business  Systems  and 
Technology  Development),  Washington,  DC 


Department  of  the  Army 

Army  Materiel  Command,  Alexandria,  VA 

Systems  Integration  and  Management  Activity-East,  Chambersburg,  PA 
Systems  Integration  and  Management  Activity-West,  St.  Louis,  MO 
Foreign  Military  Sales  Automation  Project  Management  Division,  U.S.  Army 
Security  Assistance  Command,  Alexandria,  VA 
Security  Assistance  Functional  Support  Office,  St.  Louis,  MO 
U.S.  Army  Security  Assistance  Command-New  Cumberland,  PA 
Security  Assistance  Management  Directorate,  Armament,  Munitions,  and  Chemical 
Command,  Rock  Island,  IL 

International  Logistics  Directorate,  Aviation  and  Troop  Command,  St.  Louis,  MO 
Security  Assistance  Management  Directorate,  Communications-Electronics 
Command,  Fort  Monmouth,  NJ 

Security  Assistance  Management  Directorate,  Missile  Command, 

Redstone  Arsenal,  AL 

Security  Assistance  Center,  Tank-Automotive  Command,  Warren,  MI 
Auditor  General,  Department  of  the  Army,  Alexandria,  VA 


Department  of  the  Air  Force 

Deputy  Under  Secretary  of  the  Air  Force  for  International  Affairs,  Washington,  DC 


Defense  Organizations 

Defense  Security  Assistance  Agency,  Washington,  DC 

Regional  Office,  Defense  Criminal  Investigation  Service,  St.  Louis,  MO 

Joint  Logistics  Systems  Center,  Wright-Pattefson  Air  Force  Base,  OH 


Non-Defense  Federal  Organizations 

General  Services  Administration,  Washington,  DC 
Regional  Office,  Fort  Worth,  TX 
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Appendix  H.  Organizations  Visited  or  Contacted 


Non-Government  Organizations 

Computer  Data  Systems,  Incorporated,  Rockville,  MD 
Computer  Data  Systems,  Incorporated,  St.  Louis,  MO 
Computer  Data  Systems,  Incorporated,  Fort  Worth,  TX 
OAO  Corporation,  Greenbelt,  MD 
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Appendix  I.  Report  Distribution 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition  and  Technology 
Director,  Defense  Logistics  Studies  Information  Exchange 
Under  Secretary  of  Defense  (Comptroller) 

Deputy  Under  Secretary  of  Defense  (Comptroller/Management) 

Deputy  Under  Secretary  of  Defense  (Comptroller/  Program/Budget) 
Assistant  Deputy  Under  Secretary  of  Defense  (Logistics  Business  Systems  and 
Technology  Development) 

Assistant  to  the  Secretary  of  Defense  (Public  Affairs) 


Department  of  the  Army 

Assistant  Secretary  of  the  Army  (Financial  Management) 

Assistant  Secretary  of  the  Army  (Installations,  Logistics,  and  Environment) 
Assistant  Secretary  of  the  Army  (Research,  Development,  and  Acquisition) 
Commander,  Army  Materiel  Command 

Commander,  Armament,  Munitions,  and  Chemical  Command 
Commander,  Aviation  and  Troop  Command 
Commander,  Communications-Electronics  Command 
Commander,  Missile  Command 
Commander,  Tank-Automotive  Command 
Commander,  U.  S.  Army  Security  Assistance  Command 
Director,  Systems  Integration  and  Management  Activity 
Auditor  General,  Department  of  the  Army 


Department  of  the  Navy 

Comptroller  of  the  Navy 

Auditor  General,  Department  of  the  Navy 


Department  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 
Deputy  Under  Secretary  of  the  Air  Force  (International  Affairs) 

Auditor  General,  Department  of  the  Air  Force 


Other  Defense  Organizations 

Director,  Defense  Contract  Audit  Agency 
Director,  Defense  Logistics  Agency 
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Appendix  I.  Report  Distribution 


Director,  National  Security  Agency 

Inspector  General,  National  Security  Agency 
Director,  Defense  Information  Systems  Agency 
Director,  Defense  Security  Assistance  Agency 
Inspector  General,  Central  Imagery  Office 


Non-Defense  Federal  Organizations  and  Individuals 

General  Services  Administration 
Office  of  Management  and  Budget 

Technical  Information  Center,  National  Security  and  International  Affairs  Division, 
General  Accounting  Office 

Chairman  and  ranking  minority  member  of  each  of  the  following  congressional 
committees  and  subcommittees: 

Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Services 
Senate  Committee  on  Governmental  Affairs 
House  Committee  on  Appropriations 

House  Subcommittee  on  National  Security,  Committee  on  Appropriations 
House  Committee  on  Government  Reform  and  Oversight 
House  Subcommittee  on  National  Security,  International  Affairs,  and  Criminal 
Justice,  Committee  on  Government  Reform  and  Oversight 
House  Committee  on  National  Security 
Honorable  Jim  Talent,  U.S.  House  of  Representatives 
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Part  IV  -  Management  Comments 


Defense  Security  Assistance  Agency  Comments 
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Defense  Security  Assistance  Agency  Comments 


Defense  Security  Assistance  Agency  Comments  on 
DoDIG  Draft  Audit  Report,  "Army  Procurement  and  Contract 
Administration  Practices  on  Computer  Software  Service 
Contracts,"  draft  dated  December  6,  1994 
(Project  No.  4CP-5004) 


RECOMMENDATIONS  FOR  CORRECTIVE  ACTION: 

FINDING  1 • a •  Discontinue  funding  for  the  future  development  of 
the  Security  Assistance  Automation,  Army,  information  system  in 
accordance  with  Joint  Logistics  Systems  Center  policies. 

DSAA  RESPONSE:  Concur.  During  the  FY  1995  budget  process,  DSAA 
disapproved  funding  for  all  but  maintenance  of  SA3.  This  action 
is  complete. 

FINDING  l.b.  Withhold  funding  for  future  maintenance  of  the 
Security  Assistance  Automation,  Army,  information  system  until 
the  U.S.  Army  Security  Assistance  Command  identifies  and 
justifies  only  essential  maintenance  expenditures  of  software  and 
hardware  requirements  for  the  Security  Assistance  Automation, 
Army,  information  system  in  accordance  with  DoD  Instruction 
8120.2,  "Automated  Information  Systems  Life-Cycle  Management 
Process,  Review,  and  Milestone  Approval  Procedures,”  and  Army 
Regulation  25-3,  "Life-Cycle  Management  of  Information  Systems." 

DSAA  RESPONSE:  Concur.  The  Army  funding  for  SA3  has  been 
reduced  to  only  essential  maintenance.  This  action  is  complete. 


FINDING  l.c.  Review  and  validate  requirements  and  budget 
documentation  on  the  Security  Assistance  Automation,  Army, 
information  system  to  determine  whether  all  available 
alternatives  and  costs  are  identified  to  meet  user  needs  and 
recommend  how  the  U.S.  Army  Security  Assistance  Command  should 
proceed. 

DSAA  RESPONSE:  Concur  with  exception.  The  responsibility  of 
identifying  and  reviewing  alternatives  and  costs  for  system 
development  and  identifying  and  meeting  user  needs  is  a 
Department  of  the  Army  responsibility.  DSAA  has  oversight  over 
the  program  and  systems  used  to  accomplish  the  management 
functions.  We  concur  that  this  action  should  be  accomplished, 
but  it  should  be  assigned  within  the  Department  of  the  Army. 

FINDING  l.d.  Establish  internal  control  procedures  to  verify 
that  the  U.S.  Army  Security  Assistance  Command  adheres  to  DoD  and 
Army  life-cycle  management  regulations  and  accounts  for  funds 


Final  Report 
Reference 


Revised 
Recommen¬ 
dation  l.c. 


71 


Defense  Security  Assistance  Agency  Comments 


Final  Report 
Reference 


Page  27 


expended  on  the  Security  Assistance  Automation,  Army,  information 
system. 

DSAA  RESPONSE:  Concur,  Within  90  days  of  finalizing  the  audit 
report,  DSAA  will  require  the  Army  to  provide  quarterly  reports 
on  the  life-cycle  management  and  status  of  funds  for  SA3 . 

OTHER  EDITORIAL  COMMENTS  AND/OR  CORRECTIONS: 

Reference  Page  29,  Paragraph:  "Guidance  Limiting  Further 
Development,  In  April  1990  1980,  DSAA  issued  guidance  , , - " 
(Change  as  indicated) 

DSAA  COMMENT:  This  memo  was  issued  on  8  April  1980  and  was 
subsequently  included  in  most  DSAA  Comptroller  budget  calls  from 
1981  onward. 
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Department  of  the  Army  Comments 


DEPARTMENT  OF  THE  ARMY 

OFFICE  OF  THE  DEPUTY  CHIEF  OF  STAFF  FOR  LOGISTICS 
WASHINGTON,  DC  20310-0500 


so-  m 


DALO-SAA 


14  February  1995 


MEMORANDUM  THRU  / 

/  VJ 

DEPUTY  CHIEF  OF  JSTAFF  FOl 


DIRECTOR  OF  THE  ARMY  STAFF 


■Ju  »<" 


XSTICS 
IS  hi</s 


ASSISTANT  SECRETARY  OF  THE  ARMY  ( INSTAIXAJJOtfS ,  LOGISTICS  AN£ric A. Of**  ^ * 
ENVIRONMENT)  .. 

OASAOA^ 

FOR  INSPECTOR  GENERAL,  DEPARTMENT  OF  DEFENSE  (AUDITING) 

SUBJECT:  IG  DoD  DRAFT  Audit  Report  on  Army  Procurement  and 

Contract  Administration  Practices  on  Computer  Software  Contracts 
(Project  No.  4CF-5004) — INFORMATION  MEMORANDUM 

1.  This  is  in  response  to  USAAA  memorandum  of  9  January  1995 
(Tab  A)  ,  which  asked  ODCSLOG  to  respond  to  your  memorandum  of 
6  December  1994  (Enel  to  Tab  A)  .  Your  memorandum  requested 
that  ODCSLOG  formulate  an  Army  position  on  IG  DoD  DRAFT  Audit 
Report  on  Army  Procurement  and  Contract  Administration  Practices 
on  Computer  Software  Contracts  (Project  No.  4CF-5004). 

2.  The  Army's  position  on  the  IG  DoD  DRAFT  Audit  Report  on  Army 
Procurement  and  Contract  Administration  Practices  on  Computer 
Software  Contracts  (Project  No.  4CF-5004)  is  at  Tab  B. 


2  Ends 


'■TRANK  S.  BESSON  III 
Director  of  Security  Assistance 


CF: 

VCSA 

ASA  (FM&C) 

CDR,  AMC 
SAAG-PRF-E 

AMC,  AMCIR-A,  Mr.  Kurzer,  274-9025 
USASAC,  AMSAC-SI,  Mr.  Haskins,  977-7389 


Peter  Liszewski/X50390 


Protective  marking  is  removed 
when  separated  from  enclosure  (e 

£>-^<3  (/ _ -n 

TYA&MLI  *<r. V*  ■  >  ",rr* 
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Department  of  the  Army  Comments 


DEPARTMENT  OF  THE  ARMY 
HEADQUARTERS.  US.  ARMY  MATERIEL  COMMAND 
5001  EISENHOWER  AVENUE.  ALEXANDRIA.  VA  22533  -  0001 


AMCIR-A  (36 -2b) 


8  February  1995 


MEMORANDUM  FOR  MR.  PETER  LESZEWSKI,  OFFICE  OF  THE  DEPUTY  CHIEF  OF 
STAFF  FOR  LOGISTICS,  DEPARTMENT  OF  THE  ARMY, 
WASHINGTON  D.C.  20310-0500 

SUBJECT:  Department  of  Defense  Inspector  General  Draft  Report, 

Army  Procurement  and  Contract  Administration  Practices  on 
Computer  Software  Service  Contracts  (AMC  No.  D9409) 


1.  We  are  forwarding  our  position  on  subject  report  IAW  AR  36-2. 
Recommendations  addressed  to  the  Commander,  U.S.  Army  Security 
Assistance  Command  are  at  Enclosure  1.  Recommendations  addressed 
to  the  Director,  AMC  Systems  Integration  and  Management  Activity 
are  at  Enclosure  2. 

2.  Point  of  contact  for  this  action  is  Mr.  Robert  Kurzer, 

(703)  274-9025. 

3.  AMC  --  America’s  Arsenal  for  the  Brave. 


RKY1^- McCOY  ^ 
Major  General,  USA 
Chief  of  Staff 
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Department  of  the  Army  Comments 


DEPARTMENT  OP  THE  ARMY 

Uft.  AH  MY  MCUIVTY  ASSISTANCE  COMMAND 
■001  EISENHOWER  AVENUE 
ALEXANDRIA,  VA  Dimooi 


AMS AC -IK  ( 36~2b) 


1  Fsbruary  1995 


MEMORANDUM  FOR  Chief,  Internal  Review  and  Audit  Compliance 
Office,  u.S.  Army  Materiel  Command 

SUBJECT*  Department  of  Defense  Inspector  General  (DODIG)  Draft 
Proposed  Audit  Report |  Project  Number  4CF-5004;  Army  Procurement 
and  Contract  Adminietration  Practices  on  Computer  Software 
Service  Contracts  (AMC  No.  D9409) 


1-  R®*«reac«  DODIG  Draft  of  a  proposed  Audit  Report,  6  December 
1994,  subject  I  Army  Procurement  and  Contract  Administration 
Practices  on  Computer  Software  Services  Contracts. 

2.  The  u.s.  Army  security  Assistance  Command  concurs  with  the 
recommendations  of  the  audit,  but  disagrees  with  many  of  the 
findings  and  statements  contained  in  the  body  of  the  report.  My 
major  areas  of  disagreement  arei 

_ _  “*  statement  that  the  Security  Assistance  Automation, 

Army  (SA3)  system  does  not  fully  satisfy  miesion  and  user 
requirements  is  incorrect.  The  SA3  is  fully  operational.  A  data 
call  to  the  system  users  in  October  1994  shows  this  to  be  true. 
Cases  nave  been  prepared  in  SA3  since  1908  and  management  modules 
8inc8  1987.  As  an  element  of  foreign  policy, 
security  assistance  responds  to  the  flux  of  world  politics  at  the 


....  4 _ -  .  -  mi utaiiaatuuHni.  mouuiNl 

•8X!!Ji>?8n«fn.u?8  8inc8  1987.  As  an  element  of  foreign  policy, 

aaslstaneo  responds  to  the  flux  of  world  politics  at  the 
national,  international  and  country  levels «  as  an  information 
system  SA3  responds  to  both  the  fluidity  of  security  assistance 
policy  and  the  constant  changes  to  information  management  systems 
i^°?Snn«^SSV8H??8?t  f8E“l«tion  and  technological  change,  since 
1^9  inception,  8A3  System  development  was  impacted  by  ADP 
acquisition  reform,  technological  change.  Army  Automatic  Data 
Proce8 fling  architectural  changes  and  fluctuations  in  funding  from 
the  Defense  Becurity  Assistance  Agency.  Mission  and  user  9 
requirements  have  also  constantly  changed  in  this  dynamic 

ohan98d  «  accommodate  all  changes.  The 
th  t  *•••  does  not  fully  satisfy 

mission  flnd  tlder  requirements •  does  not  fully  moot  us  or 
needs...  and...  was  wasteful,*  is  not  substantiated  in  the 
report.  I  believe  SA3  satisfies  its  initial  focus  of  providing  a 

information  “i  !y8t81B  wit?  raPid  »cces»  to  currentf 

0W  currently  capable  of  preparing 

100  percent  of  the  Amy's  new  foreign  Military  Sales  (fMS)  cases 
because  it  has  been  amended  whenever  possible  to  incorporate  user 
co^nd  Un«?U  raent8  Wh8n  tb8y  pro^d  b8»-«ioial  ecSo.e 


Page  10 


f  A  ~L  J~ 
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Department  of  the  Army  Comments 


Final  Report 
Reference 


Page  14 


AMSAC*-IM  (36-2b) 

SUBJECTi  Department  of  Dofania  Inspector  General  (DOOIG)  Draft 
Proposed  Audit  Report;  Project  Number  4CP-5004;  Army  Procurement 
and  Contract  Admin lit rat ion  Practices  on  computer  Software 
Service  contracts  (AMC  No.  D9409) 


to.  TIM  draft  report  move,  batwuen  hi.tory  and  the  present 
failing  to  identify  whether  it  is  addressing  the  old  S2X  data 
base  or  newer  M204  data  base  version  of  SA3.  As  initially 
conceived,  the  Sa3  was  primarily  a  standardized  and  centralized 
preparation  system  that  provided  case  management  tools. 
***^kth©  availability  of  M2  04,  the  Army  could  use  the  data 
residing  in  SA3  for  other  applications  such  as  the  Security 
^1!  e!!£an?e  Acquisition  Program.  The  improvements  made 

to  $A3  Since  FY90  should  not  be  linked  to  SA3  for  life  cycle 
management  purposes.  They  are  specialised  applications  and 

^”ETlt:H*h“,Wara  P®!! 1)514  b?c4u4*  Of  the  outgrowth  of  M204 
advances.  The  6A3  currently  provides  a  uniform  process  for  case 
development,  a  set  of  tools  for  case  management,  and  a  repository 
of  dats  that  is  both  currsnt  and  accurate,  it  serves  the  1 

customers  of  Army  FMS  well  and  has  proved  its'  vaius  tinder  crisis 
situations.  Examples  of  benefits  of  SA3  ares 

.  .  *  ,_Th«  SA3  took  security  assistance  out  of  the  automation 

dark  ages  by  providing  three  generations  of  automation  equipment 

liIlTL  u0?  individuals  in  the  security  assistance  process  in 
th©  Army  Materiel  Command. 

,  .  .  '  During  Desert  shield  and  Desert  storm  FMS  cases  were 

tasked,  written,  electronically  transmitted,  signed  and  ready  to 
go  to  DSAA  within  S  hours.  J 

•  Tha  SA3  provides  one>stop,  one  data  element  entrv 
access  to  FMS  case  logistical,  financial  and  procurement 
Information  contained  in  Commodity  Command  standard  System  files. 
Prior  to  SA3  access  to  this  information  required  multiple  file 
accesses  using  a  number  of  different  access  methods. 

•  The  SA3  provides  communications  connectivity  for 

electronic  data  transmission  and  electronic  mail  capability 
through  the  Internet.  ' 

•  Tha  SA3  established  the  first  dedicated  facsimile 
network  for  security  assistance  which  grew  to  over  100  machines 

«vingWprSj«tMdry?  Y“ar“  “d  U  mtL11  *  vl,ble  “°n4y 


2 
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Department  of  the  Army  Comments 


AMSAC-IM  ( 36-2b ) 

SUBJECTS  Department  of  Defense  Inspector  General  (DODIG)  Draft 
Proposed  Audit  Report)  Project  Number  4CF-5004/  Army  Procurement 
and  Contract  Administration  Practices  on  Computer  Software 
Service  Contracts  (AMC  Vo.  D9409) 


.  .  '  Th*  SA3  was  (and  continues  to  be)  a  leader  In  systems 

and  functional  training,  providing  Computer  Based  Training  for 
modules  of  SA3  as  well  as  other  security  assistance  functions, 

c.  The  SA3  system  development  was  initiated  in  FY82  and 
completed  by  FY90.  The  system  was  fielded  and  in  February  1991 
the  Project  Manager  position  was  abolished.  An  6A3  maintenance 
team  was  retained  to  ensure  maintenance  of  the  system, 
integration  of  Army  directed  -state-of-the-art*  technological 
Improvements  to  the  System,  and  oversight  o£  system  improvements 
requested  by  the  users.  Yet  the  audit  team  declared  SA3  to  etill 
b*  ?Mign  *nd  development  status  and  addressed  their  findings 
without  regard  to  the  fact  that  the  system  has  been  fielded  for 
over  3  years.  This  Increased  the  cost  figure  of  $46  million 
reported  for  the  development  of  the  system  and  understates 
capabilities  of  the  system  since  it  is  not  considered  fielded. 

^  J  hope  that  the  Draft  Report  oan  be  revised  to  accommodate 
the  information  provided  by  the  USASAC  system  managers 
(Enclosure),  The  developers  of  a  A3  did  a  good  job  and  the  system 
maintainors  Aro  also  doing  dxcellent  work  to  insure  that  the 
system  can  be  used  to  support  Army  managers  and  the  FMfi 
customers.  My  plan  to  accommodate  the  recommendations  of  the 
report  follows  i 

a.  Recommendation  2. a.  Establish  procedures  to  verify  that 

approval  of  software  changes  for  the  BA3  information  system  and 
that  tracking,  updating  and  reporting  of  costs  associated  with 
the  system  comply  with  Army  Regulation  25-3,  '’Life-Cycle 
Management  of  Information  Systems.-  1 

CONCUR  -  A  senior  level  review  committee  will  be  established 
to  provide  oversight  of  security  assistance  information  systems. 
Target  date  for  establishing  committee*  April  1995.  y 

b.  Recommendation  2.b.  Periodically  review  the  SA3  infor¬ 

mation  system  as  s  control  technique  to  determine  compliance  with 
dod  instruction  8120,2,  "Automated  Information  Systems  Life-Cycle 
Management  Process  Revisw,  and  Milestone  Approval  Procedures,* 
and  Army  Regulation  25—3,  "Life- Cycle  Management  of  Army 
Information  Systems."  3 
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Page  80* 


’“Partially  omitted  because  of  length.  Copies  will  be  provided  upon  request. 
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AMSAC-IM  (36-2b) 

subjects  Department  of  Defense  Inspector  General  (DODIG)  Draft 
Proposed  Audit  Report;  Project  Number  4CF-5004;  Army  procurement 
and  contract  Administration  Practices  on  Computer  Software 
Service  Contracts  (AMC  No*  D9409) 


CONCUR  -  The  senior  level  review  committee  will  assess 
compliance  semi-annually.  Target  date  for  first  reviews  July 
1995. 

c.  Recommendation  2.c.  Evaluate  the  SA3  information 
system  to  identify  total  user  software  and  hardware  require¬ 
ments,  including  personnel  required  to  support  the  system. 

CONCUR  -  A  Business  Case  and  Functional  Economic  Analysis 
for  SA3  will  be  prepared.  Software  has  been  requested  from  the 
Defense  Information  Systems  Agency.  All  project  actions  will  be 
tracked  using  a  PC  based  project  management  program,  already 
available.  Target  date  for  completion  of  Business  Cases  June 
1995.  Target  date  for  completion  of  FEAs  August  1995. 

d.  Recommendation  2.d.  Based  on  the  evaluation  of  the  8 A3 
information  system  performed  in  recommendation  c.  above, 
determine  the  estimated  cost  to  complete  the  system,  the 
estimated  cost  to  maintain  the  system,  and  the  expected  benefits 
and  cost  reductions  that  will  be  achieved  when  the  system  is 
completed.  This  information  should  be  included  as  part  of  the 
command  annual  budget  submission  to  the  Defense  Security 
Assistance  Agency. 

CONCUR  -  The  information  will  be  obtained  from  the  Functional 
Economic  Analysis.  It  will  be  used  for  the  FY96  budget 
submission.  Target  dates  August  1995. 

e.  Recommendation  2.e.  Disestablish  the  FMS  Automation 
Project  Design  Field  Office,  St.  Louis,  Missouri,  as  it  adds  no 
value  to  the  SA3  information  systems  program  function  and 
develops  functions  that  can  be  adequately  provided  by  the  Systems 
Integration  and  Management  Activity,  U.S.  Army  Materiel  Command. 

CONCUR  -  Alternatives  will  be  developed  for  disestablishment 
of  the  field  office.  Target  Date  for  developing  alternatives! 

May  1995.  Target  Date  for  disestablishing  office*  October  1995. 
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AMSAC-IM  ( 36-2b) 

SUBJECT!  Department  of  Defense  Inspector  General  (DODIG)  Draft 
Proposed  Audit  Report?  Project  Humber  4CF-5004?  Array  Procurement 
and  Contract  Administration  Practices  on  Computer  Software 
Service  Contracts  (AMC  No.  D9409) 

4*  If  additional  information  ie  needed,  please  contact 
Mr.  Alfred  D.  Haskins,  AMSAC-SI,  DSN  977-73B9/5133 „ 


/lUW&W — 

MICHAEL  S.  DAVISON,  JR. 
Major  General,  USA 
Commanding 
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U.S.  Army  Security  Assistance  Command 
Information  Management  Directorate 
FMS  Automation  Project  Management  Division 


Results  and  Analysis  of  Security  Assistance 
Automation,  Army  (SA3) 

Data  Call 

OCT  -  DEC  1994 


Chester  Jay  Freedenthal 
DSN  284-4018 

email  cfreed@alexandrla-emh6.army.mil 
January  1995 

Enclosure  to  Enclosure 
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Department  of  the  Army  Comments 


V  SA3D*t*  Call  Oct-Dec  1994 


SUMMARY 

In  reviewing  a  working  draft  of  an  audit  by  the  DOD1G  it  became  apparent  that 
many  of  the  conclusions  of  the  IG  were  inaccurate.  These  inaccuracies  led  to  findings  and 
recommendations  that  needed  to  be  refuted. 

I  developed  a  data  call  of  13  questions  for  each  of  the  SA3  user  sites  and 
requested  they  be  as  candid  as  possible  in  their  response.  I  identified  the  purpose  of  the 
data  call  and  assured  them  that  all  responses  would  be  included  in  a  summary,  but  that  no 
identification  would  be  placed  with  the  responses.  The  data  call  was  issued  in  October 
and  it  took  until  early  December  for  all  responses  to  be  received.  Numeric  responses  were 
tabulated  and  averaged  and  text  responses  were  used  verbatim. 

This  document  is  a  compendium  of  all  responses.  It  provides  the  question,  the 
responses  and  a  short  analysis  and  comment  section  for  each  question. 

The  responses  were  favorable  to  SA3  and  provide  substantial  evidence  about  the 
inaccuracy  of  many  of  the  IG’s  conclusions  and  subsequent  findings.  There  were  no  real 
surprises  and  in  many  instances  the  weaknesses  perceived  by  the  users  arc  being 
addressed.  This  data  call  can  be  used  as  baseline,  and  with  slight  modification,  could  be 
used  every  year  to  gauge  the  level  of  customer  satisfaction  how  well  SA3  is  doing. 
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Department  of  the  Army  Comments 


SA3QatM  Call  Oct-Dec  1994 


1.  At  the  current  time,  what  percentage  of  the  following  documents  are  prepared 
using  SA3?  (  7  sites  reporting) 

LOA  98.4% 


Modifications  79.7%  (92.7%  without  USASAC  2%) 

Amendments  79.7%  (92.7%  without  USASAC  2%) 


ANALYSIS  AND  COMMENT  -  These  figures  show  that  that  all  sites  are  using  SA3 
successfully.  There  are  two  significant  reasons  LOAs  may  not  be  prepared  in  SA3 ; 
the  LOA  may  be  classified,  or  the  user  decides  notto  use  the  system.  Modifications  and 
amendments  not  prepared  in  SA3  mostly  result  from  the  basic  LOA  or  previous 
modifications  or  amendments  not  being  in  SA3  and  the  user  not  desiring  to  load  them. 
This  often  occurs  on  old  LOAs  and  when  simple  modifications  are  done  during  ca^e  close¬ 
out.  USASAC  has  only  been  preparing  basics  for  a  short  time;  therefore  they  have  limited 
use  of  modifications  or  amendments  unless  they  back  load.  This  is  why  two  figures  are 
shown  for  modifications  and  amendments. 
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Department  of  the  Army  Comments 


2.  What  would  you  estimate  to  be  the  percent  Increase  in  your  commands  use  of 
SA3  ( 7  sites  reporting) 


over  the  last  6  months? 


0  to  73% 


over  the  last  year? 
over  the  last  2  yean? 


0  to  95% 


0  to  100% 


ANALYSIS  AND  COMMENT  -  Although  somewhat  confusing  when  arrayed,  the 
responses  were  as  expected.  Users  responses  were  indicative  of  the  level  of 
experience  with  the  system  in  LOA  preparation.  More  experienced  sites  showed  0  to  a 
small  percent  increase  over  the  last  year.  Inexperienced  sites  showed  medium  to  large 
increases  over  the  last  2  years,  and  somewhat  smaller  increases  over  the  last  6  months. 
This  shows  that  as  users  become  more  familiar  and  comfortable  with  the  system,  they  use 
it  more.  This  data  and  the  data  for  question  1  validates  that  the  system  is  used  and  satisfies 
mission  requirements  for  case  development 
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3.  Do  you  feel  that  SA3  has  helped  to  reduce  case  development  time  at  your 
command?  If  yes,  please  estimate  by  how  many  days.  If  no,  please  explain  why  you 
believe  H  has  not  (7  sites  reporting) 

Yes,  total  time  around  7  days.  SA3  has  also  eliminated  the  mail  time  to  USASAC  It 
has  also  reduced  time  to  make  revisions  to  cases. 

Yes,  definitely.  Most  managers  enter  data  as  they  gather  it  Especially  since  we  have 
cut  and  paste  features  and  are  able  to  pull  data  from  similar  or  model  cases.  Days  aze  hard 
to  estimate  since  cases  come  in  such  different  sizes,  complexities  and  variations. 

SA3  has  helped  reduce  case  development  time  by  5-7  days  when  the  system  is  fully 
operational.  Does  not  account  for  downtime  of  laser  printers.  Another  measure  is  that 
development  time  is  reduced  50%  after  receipt  of  P&A  data. 

Have  saved  1  day  in  prep  time,  not  including  time  saved  in  mailing. 

Some  users  responded  a  definite  yes  and  they  estimate  anywhere  from  4  to  20  days 
savings  in  time  to  prepare  a  case.  Other  users  say  no,  citing  downtime  and  print  problems 
as  impeding  their  preparing  a  case. 

Yes.  Prior  to  SA3  cases  were  hand-written  and  forwarded  for  typing  by  secretarial 
staff  consisting  of  1  or  2  secretaries  for  25-30  people.  This  created  a  backlog.  With  the 
implementation  of  SA3,  submission  is  still  driven  by  item  manager's  price  and  availability 
and  other  directorate's  input,  but  each  case  writer  is  responsible  for  their  own  case 
preparation-  the  actual  LOA  preparation  time  is  between  4-7  days  for  the  novice  writer. 

Yes.  Conservatively  it  saves  5-7  days  processing  time.  Each  case  writer  prepares  the 
final  LOA,  mod  or  amend  to  transmit  to  USASAC  The  alternative  is  to  have  one  or  two 
secretaries  typing  for  all  27  case  writers. 

1  ■  ^ 

ANALYSIS  AND  COMMENT  •  One  of  the  goals  of  SA3  was  to  reduce  case 
development  time.  The  responses  show  an  average  saving  in  LOA  preparation  of 
about  20%  of  the  total  allowed  time.  Additional  time  is  saved  by  eliminating  mail  time  ( 1- 
3  days );  mail  costs  are  also  eliminated.  Recent  industry  studies  show  that  the  cost  of 
electronic  transmission  is  one  1/3  that  of  regular  mail.  The  automated  system  reduces 
typing  workload  as  welL  Not  included  in  the  responses  is  the  significant  reduction  in 
preparation  time  for  modifications  and  amendments.  In  some  instances  the  time  has 
dropped  to  less  than  one  work  day  for  development,  approval  and  transmission. 
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4.  What  Is  the  most  significant  reason  for  not  using  SA3  to  develop 
a  basic  LOA? 

Emergency  requirement  when  SA3  is  down 

Case  is  classified  -  2l£SPgDS£S 

None  llCSPOnggg 

System  Downtime 

Lack  of  on-line  edits,  i.c.  MASL 

...  a  modification? 

Basic  case/mod/amend  not  in  SA3.  6  responses 

Previous  Mod/Amend  not  implemented. 

If  case  is  in  close  out  and  there  arc  too  many  lines  to  back  load.  3  responses 
M.  an  amendment? 

None 

Basic  case/mod/amend  not  in  SA3.  6  responses 

Previous  Mod/Amend  not  implemented. 

If  case  is  in  close  out  and  there  arc  too  many  lines  to  back  load.  3  responses 


'W  - 

ANALYSIS  AND  COMMENT^  7  sites  reporting;  similar  responses  from  multiple 
sites  identified.  Some  sites  reported  more  than  one  reason.)  SA3  can  and  does 
perform  well  and  satisfies  mission  requirements.  These  response  validate  the  high 
percentage  of  use  identified  in  question  1.  The  only  significant  reasons  arc  the  case  is 
classified  (S A3  was  never  designed  to  process  classified  data)  or  the  basic  LOA, 
modification  or  amendment  is  not  in  SA3.  This  could  be  because  of  the  age  of  the  case, 
to  many  modifications  or  amendments  to  load  ,  or  changes  required  on  closed  cases,  or 
some  other  reason.  System  down  time  is  a  legitimate  response,  but  is  not  a  factor  of  the 
system,  but  outside  influence  on  the  system.  Most  downtime  is  experienced  as  a  result  of 
mainframe  hardware  downtime  rather  than  SA3  system  downtime. 
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SAB  Data  Call  Oct-Dac  1994 


5.  If  SA3  is  not  used,  what  method  do  you  use  for  case  development? 

Word  Perfect  2  responses 

Program  created  In-house 

Sperry  Word  Processing  prototype  program.  Most  case  writers  and  secretaries  arc 
unfamiliar  with  it  and  case  prep  time  doubles.  *  2  responses 

Multimate 

A  PC  based  LOA  format  (Program  not  identified)  2  responses 

A"1"--  ^ . 

NALYS1S  AND  COMMENT-  This  was  an  information  gathering  question  to 
determine  what  was  being  used  as  a  case  development  alternative,  As  expected,  all 
responses  indicated  either  a  PC  based  or  mini-computer  UNIX  based  alternative. 

What  is  significant  is  that  there  is  such  a  divergence  among  the  sites  as  to  the  tool  used. 
The  automated  system  provides  a  standard  means  of  case  preparation  at  all  sites  as  well  as 
a  means  of  storing  information  in  a  database  for  easy  manipulation  of  the  data  for  changes. 
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6.  How  extensive  Is  the  use  of  the  SA3DOCU  file  at  your  command?  If  low,  what  Is 
It  that  inhibits  Its  use?  If  high,  what  Is  It  that  makes  It  usable? 

Used  by  financial  and  case  close-out  people.  Not  used  as  extensively  as  it  should  be. 

Moderate.  Most  users  use  "CISIL"  or  internal  CCSS  document  control  file  (DCF)  or 
REACT  for  requisition  status.  It  is  used  as  a  historical  record  when  files  have  been  purged 
in  CCSS. 

Not  used  extensively  because  of  lack  of  information  and  training. 

Low  usage.  Users  generally  use  DCF  or  REACT  for  requisition  status,  comparing 
the  data  to  CISIL.  In  the  past  much  of  the  data  has  seemed  to  be  incomplete  and/or 
incorrect  and  there  is  not  a  great  deal  of  confidence  in  the  file.  As  a  historical  record  it  is 
used  when  the  DCF  has  been  purged. 

Users  were  not  familiar  with  the  name  SA3DOCU  -  everyone  said  they  did  not  use  it 
However  it  is  available  for  their  use.  They  sometimes  use  this  option,  but  not  often, 
saying  the  data  is  unreliable. 

Wc  do  not  know  what  it  is.  If  it  is  there  to  help  in  case  development  then  its 
capabilities  should  have  been  identified  and  promoted  to  the  field. 


ANALYSIS  AND  COMMENT-  The  response  to  this  question  showed  that  we  had 
not  educated  our  users  successfully  on  what  SA3DOCU  is  and  bow  it  is  used.  We 
have  developed  a  CBT  course  for  SA3DOCU  which  will  be  called  Security  Assistance 
Logistics  Data.  It  will  be  ready  for  users  in  February  1995.  When  wc  moved  from  System 
2000  (S2K)  data  base  to  the  Model  204  (M204)  data  base  we  combined  our  old  SAFD 
and  SAAD  files  into  the  SA3DOCU  files.  This  caused  some  confusion.  Although  we 
provided  documentation  and  limited  training  on  its  purpose  and  use,  it  did  not  get  to  the 
users  as  it  should  have.  Our  SA3DOCU  files  contain  replicated  CCSS  data  in  a  more 
accessible  format  Users  show  a  reluctance  to  use  it  because  they  doubt  the  credibility  of 
CCSS  data  in  general,  or  because  the  SA3  files  are  updated  less  frequently  at  some  MSCs 
(  anywhere  from  multi-daily  at  one  MSC  to  twice  a  week  at  another .)  Many  users  stay 
with  old  ways  and  refuse  to  access  the  data  in  any  way  other  than  bow  they  used  to  access 
it  prior  to  SA3.  The  SA3DOCU  file,  unlike  CCSS,  is  never  purged  of  data  and  so 
provides  an  excellent  audit  trail  for  logistics  transactions.  Those  users  familiar  with  SA3 
noted  this  as  a  distinct  advantage. 
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SA3  Datm  Cmlf  Oct-Dmc  1994 


7.  Do  you  use  SAMAP7  If  so,  bow  effective  have  you  found  It?  If  not,  why  not, 
what  are  the  significant  weaknesses?  ( 7  sites  reporting) 

No.  Have  not  been  able  to  run.  Wc  are  willing  to  give  it  a  try.  Wc  are  currently 
working  to  get  functional  help  from  another  MSC. 

Vcs  but  very  little.  We  have  no  comment  as  to  weakness  because  of  little 
knowledge  of  the  system.  We  are  slowly  starting  to  use  it  The  CBT  has  been  made 
available  to  the  case  managers  and  it  is  being  pushed  to  be  used. 

Wc  have  approximately  50  cases  loaded.  Wc  have  not  bad  the  resources  to  train  all 
users.  Plans  arc  being  made  to  train  users.  The  inability  to  report  directly  to  USASAC  has 
hindered  acceptance. 

Have  opted  not  to  use  SAMAP  due  to  problems  users  have  found. 

Not  used  extensively  due  to  lack  of  policy  and  procedures. 

Case  writers  arc  aware  to  utilize  the  SAMAP  indicator  code.  SAMAP  was  not 
usable  on  our  commodity  till  late  FY94.  Due  to  possibility  of  file  corruption  during  file 
split,  intensive  training  was  delayed. 

A"'  *w 

NALYS1S  AND  COMMENT-  Acceptance  of  SAMAP  has  been  slow,  as  indicated 
by  the  responses.  The  SAMAP  was  developed  in  response  to  USASAC  leadership 
dropping  the  ILSDP  process  and  desiring  to  create  a  new  automated  system  that  would 
provide  a  tool  for  users  and  managers  to  be  pro-active  in  the  management  of  major 
acquisitions.  It  would  also  be  the  source  of  procurement  data  for  the  USASAC  CISIL* 
since  no  other  automated  source  is  available.  SAMAP  was  designed  by  the  user’s  and 
developed  cooperatively  by  MICOM  and  SIMA.  Its  desired  capability  grew  so  fast  that 
we  had  to  break  it  into  pieces  to  field  it  If  those  pieces  desired  by  a  particular  MSC  were 
not  fielded  they  chose  to  not  use  it  The  Policy  office  would  not  work  with  us  in 
developing  rules  for  use  of  SAMAP  and  some  MSCs  chose  lo  continue  to  use  the  old 
ILSDP  even  though  it  had  been  canceled.  Its  use  was  compounded  by  the  retirement  of 
the  senior  USASAC  leader  that  spearheaded  its  creation.  We  developed  a  SAMAP  CBT 
course  and  offered  group  training  at  each  MSC.  Several  changes  have  been  made  to 
SAMAP  to  enhance  its  usability,  but  there  is  a  continuing  reluctance  to  use  it  There  needs 
to  be  a  concerted  effort  lo  publish  guidance  on  its  use  and  for  users  to  learn  how  to  use  it 
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¥ 


SAS  Data  CmH  Oct-Dec1994 


8.  What  do  you  consider  to  be  tbe  3  most  significant  benefits  that  SA3  has  provided 
for  case  development?  case  management? 

CASE  DEVELOPMENT 
Faster  5_responses 

Calculations  Automated  4  responses 

Notes  Automated 

Simplicity /User  Friendly  2  responses 

Acc  urate  AJniform /Standardized  format  5  responses 

Time  savings  of  transmission  vs.  mail  4  responses 
Has  put  control  of  input  and  data  in  case  managers 
Generation  of  Mods  and  Amends 

CASE  MANAGEMENT  - 


Case  Duplication 

Permanent  Refercnce/Visibility  to  all  case  documents  2  responses 
Document  history  ( Requisition  )  can  be  accessed  longer  than  in  CCSS  files 
Case  reports 
Not  used  at  this  MSC 


ANALYSIS  AND  COMMENT-  The  responses  say  good  things  about  SA3...  it  does 
what  it  was  designed  to  do.  The  responses  to  case  management  show  some 
misunderstanding  about  its  use.  Some  uses  see  it  as  an  extension  of  case  development 
where  it  provides  a  historical  record  of  the  case,  others  viewed  it  as  the  SA3DOCU  file 
which  is  the  way  it  was  intended.  Case  development  encompasses  all  of  the  actions 
needed  to  prepare  a  case  and  any  changes  to  it.  Case  management  is  the  retrieval,  storage 
and  analysis  of  logistics  data  supporting  the  execution  of  the  case.  This  relates  to  the 
analysis  of  question  6. 


Department  of  the  Army  Comments 


9.  What  do  you  consider  the  3  most  critical  weaknesses  of  SA3? 

Lack  QflLimitcd  Word  Processing  Features  2  responses  { We  have  provided  as 
much  word  processing  capability  as  the  system  architecture  will  allow.  We  recently  added 
a  line  insert  and  delete  capability  which  wQl  help  oa  changes  to  long  text  fields.] 

No  Cut  &  Paste  at  subline,  annex  line  [Cut  and  paste  capability  is  a 
relatively  new  feature  that  the  users  like.  This  takes  it  to  amore  detailed  level  We  arc 
looking  at  how  we  can  implement  it,  but  it  may  not  be  possible.] 

lack  of  Classified  capability  [SA3  never  was  designed  to  handle  classified  cases. 
The  small  number  of  classified  cases  ( less  than  1%  )  could  not  justify  the  tremendous  cost 
involved  in  providing  a  classified  system.] 


i  im»7T  1  Mi'Jlii  rT*?  Jkh  Tyi 


2.lgsp9nK?  [  This  relates  to  both  word  processing  and  edits.  We  cannot  include  a 
spell  checker.  Wc  are  in  a  continual  foot  race  with  users  and  policy  on  data  edits  and  code 
checks.  Some  want  more,  others  want  less,  some  want  none  at  all!  We  feel  that  wc  have  a 
manageable  amount  now  and  we  are  always  reviewing  to  see  where  changes  can  be 
made]. 

Rounding  Problem  on  FABS/FICS  Worksheet  [  We  have  addressed  this  many 
times  and  it  continues  to  be  a  spurious  problem.  The  rounding  occurs  in  a  number  of 
places  and  is  difficult  to  trace  an  error.  See  next  response.] 

Inaccuracy  of  mathematical  calculations  {  As  indicated  above,  rounding  occurs  at 
many  places  and  causes  some  minor  math  errors  when  adding  or  subtracting.  The  errors 
are  infrequent  and  can  be  corrected  manually.  We  developed  a  requirement  document  to 
totally  revamp  the  math  processes,  but  it  most  likely  will  not  get  programmed.  Where  we 
identify  critical  system  errors  we  will  change  the  code.] 

System  Downtime/Mainframe  Dependency  6  responses  [  We  have  no  control 
over  system  downtime  when  the  mainframe  is  at  fault.  When  it  is  an  SA3  problem  ,  it  gets 
top  priority  and  fixed  ASAP.  Wc  are  a  mainframe  based  system  and  do  not  have  the 
resources  to  go  to  a  PC  based  or  client  server  environment  (  both  of  which  have  inherent 
problems  of  their  own.] 

Printing  Downtime  2  responses  [  The  print  process  is  admittedly  the  weakest 

link  in  SA3.  Wc  use  old  technology  and  old  equipment,  but  we  don’t  have  the  resources  to 
change  it  We  have  excellent  technicians  that  diagnose  and  fix  problems  very  quickly. 

Over  the  last  six  months  we  have  not  had  critical  print  problems.  WE  also  use  the  systems 
remote  print  capability  as  a  back-up  for  serious  problems.] 
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nuances  ( This  appears  to  be  an  isolated  problem.  Wc  have  many  training  avenues 
available  that  (he  user  can  take  advantage  of.  Responses  to  other  questions  indicates  that 
as  users  become  more  familiar  with  the  system  they  Ond  it  easier  to  use. 


also  a  database,  we  need  complete  data  to  maintain  accuracy  and  integrity.] 


[  SA3  is 


Unable  to  page  print  from  mainfmrng  Zxcsponses  [Wc  would  like  for  this  to  be 
available,  but  the  system  architecture  does  not  permit  it  Users  can  print  pages  from  the 
UNIX  mini-computer.] 


such  a  desk  guide  in  preparation  and  it  will  be  available  in  March  1995.] 


i  [  We  have 
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used  to  develop  a  case,  but  they  may  account  for  the  2  percent  of  new  LOAs  that  are  not 
developed  in  SA3.  Each  of  them  have  been  addressed  individually  to  provide  a  more 
thorough  analysis  (comments  are  [bracketed].)  Most  of  the  weaknesses  were  going  to  be 
addressed  in  future  changes  to  the  system  as  resources  and  priorities  permitted.  This  most 
likely  will  not  happen. 
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SA3  Data  Call  Oct-Dee  1994 


10.  To  what  extent  does  your  command  use  the  AMC  Standard  FMS  Pricing 
System  (ASFPS)?  Hu  the  loss  of  the  Interface  between  SA3  and  the  ASFPS  made  a 
difference  to  you? 

Do  not  use.  2  responses 

Use  extensively.  Lack  of  interface  has  caused  uses  to  go  back  to  dual  line  item 

entry. 

We  provide  pricing  data  using  ASFPS  about  80%  of  the  time.  It  is  currently  used 
for  procurement  items,  SDAF,  excess,  DBOF  and  stocked  items  where 
price  is  constant.  Interface  was  never  functional. 

Used  by  financial  office  on  every  case.  We  do  compare  calculations  from  both 
systems  against  each  other.  In  our  case  the  pricing  system  is  used  for  case 
preparation  but  not  by  the  case  manager.  The  interface  was  never  very  reliable. 

The  ASFPS  was  established  for  items  which  are  being  procured  and  have  planned 
future  procurement.  All  items  meeting  this  criteria  arc  in  the  pricing  system  and 
data  is  used  for  pricing  the  item.  The  usage  has  been  expanded  to  indude  SDAF, 
EDA  and  DBOF  items.  Usage  also  includes  frequently  requested  items  from  stock, 
if  price  is  constant  The  interface  between  the  ASFPS  system  was  never  functional 
to  date,  therefore  no  difference. 


ANALYSIS  AND  COMMENT-  The  ASFPS  history  is  long  and  harrowing.  Bottom 
line  is  that  we  paid  a  lot  of  money  to  document  it,  re-program  it  and  get  it  installed  at 
all  sites,  only  to  have  another  activity  continue  to  develop  new  and  improved  versions. 

The  interface  was  a  mistake  to  do,  but  it  was  a  directed  action.  Had  the  original 
development  of  the  ASFPS  been  part  of  SA3  they  would  have  been  complimentary, 
instead  they  worked  at  odds  with  each  other.  The  use  of  the  system  is  varied  across  the 
MSCs  and  the  loss  of  the  interface  didn’t  mean  much.  The  major  asset  of  the  ASFPS  for 
SA3  was  the  automated  development  of  a  payment  schedule.  SA3  will  still  have  to  wait 
for  this  capability. 
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11.  How  beneficial  has  (he  SA3  Computer  Based  Training  (CBT)  courses  to  your 
command?  Do  you  use  the  SA3  on-line  help  function? 

The  M204  is  the  only  one  that  has  really  been  used.  It  was  helpful  to  those  who 
took  the  time  to  compete  it  On-line  help  functions  were  not  there  for  so  long,  it 
teems  most  people  are  unaware  they  exist. 

Some  have  taken  the  course,  but  most  users  responded  it  was  to  rime  consuming 
or  easier  to  ask  someone  else  who  knows  case  development  better  than 
themselves.  On-line  help  used  by  a  few,  unused  by  most 

Some  case  managers  have  taken  it  Others  have  said  they  just  "dived-in"  to  SA3 
without  using  CBT.  On-line  is  not  used. 

The  CBT  training  is  beneficial,  but  normally  the  functional  users  have  already  had 
hands-on  training  and  it  does  not  benefit  them.  The  SAMAP  course  will  be  an 
asset.  The  on-line  help  is  not  used  often.  It  might  be  more  useful  if  the  definitions 
included  the  code  options  or  listed  the  choices  of  responses  that  could  be  included 
in  the  field. 

Not  used.  Use  only  hands-on  training. 

The  CBT  courses  arc  very  beneficial  in  preparing  dcvclopersAnanagers  to  use  new 
system.  Responses  were  positive  and  enthusiastic.  On-line  help  is  having  a  user 
manual  at  your  finger  rips. 

The  CBT  training  is  beneficial  to  those  just  becoming  familiar  with  SA3.  Most 
functionals  have  case  writing  experience  and  have  hands-on  training  and  are 
familiar  with  the  system.  When  SAMAP  is  implemented  the  SAMAP  course  will 
be  essential  to  provide  an  overview  of  SAMAP  and  what  it  does  and  how  to 
access  it  On-line  help  function  is  not  accessed  often. 

- nr 

NALYSIS  AND  COMMENT-  These  responses  were  disappointing.  The  SA3  CBT 
courses  were  among  the  best  work  we  did.  Tbc  CBT  expands  beyond  SA3  as  well, 
including  many  security  assistance  peculiar  functional  areas.  It  appears  that  the  users  just 
weren’t  ready  for  this  type  of  training.  The  courses  are  well  done,  accurate  and 
entertaining.  Most  users  appear  to  want  hands-on  training.  The  CBT  was  actually 
designed  to  do  that,  as  they  include  considerable  simulation  as  part  of  the  instruction.  Wc 
have  found  that  many  of  the  errors  reported  by  users  could  have  been  avoided  if  they  had 
used  the  CBT.  When  used  the  CBT  provides  a  good  opportunity  for  the  user  to  know 
the  system  before  they  use  it  or  to  refresh  their  knowledge.  Tbc  on-line  help  function  is 
new  to  SA3  and  many  users  don't  know  it  exists.  It  reinforces  what  is  in  the  CBT  and  can 
be  customized  by  the  MSC  Users  need  to  be  informed  it  is  there  and  bow  to  use  it. 
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12.  How  would  you  rate  the  support  ( l*e.,  training,  customer  response,  problem 
solving)  you  have  received  over  the  years  from  the  following  activities? 

(1  «pooi%2*satlifactory  ,3  «good,4«excd  lent) 

SA3  PM  Office  (My  Office)  3.4 

St  Louis  Field  Office  (  Mr.  Keeling's  Office)  3.1 
SIMA  3.9 

Other  (  Fill  in) _  No  Responses 

(7  responses  received) 

What  do  you  feel  has  been  the  best  and  worst  services  provided  by  the  above.  ? 

Quick,  professional,  friendly  quality  service.  Has  improved  since  we  began  talking 
directly  to  the  programmers  and  functionals. 

Responses  have  been  quick  and  with  good  attitude. 

Best  is  guidance  and  technical  training.  Worst  is  occasional  breakdown  in 
communication. 

Timely  answers,  supportive,  good  corporate  knowledge.  Field  office  often  took  to 
long  lo  answer  problems  (  several  days).  Also  felt  as  if  problems  were  always 
perceived  as  user  error  first,  even  if  system  error.  Have  had  good  response  with 
contacting  programmers  or  functionals  directly.  Downside  is  problems  experienced 
with  EUCPs.  Many  times  it  seems  that  the  change  produces  additional  problems. 

Possibly  more  thorough  testing  should  be  done. 

This  is  hard  because  sometimes  we  receive  excellent  support  and  at  others  it  was 
satisfactory.  Depends  on  the  situation  and  who  you  talk  to.  The  best  service  is 
solving  SA3  problems  quickly. 

Best  is  response  by  SIMA  programmcr/functionals  on  conversion  from  S2K  to 
M2 04.  Situation  Room  wasn't  very  responsive.  Also  release  management  had 
problems  getting  out  correct  release  tape  or  EUCP  was  incorrect. 

Timely,  patient  answers.  Good  corporate  knowledge  of  database.  Always 
supportive  and  willing  to  resolve  problems.  Responsive.  Good  problem  analysis. 
Worst  is  user  manual  for  SA3  -  not  user  friendly. 
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ANALYSIS  AND  COMMENT-  These  response  reflect  the  insights  of  the  new  people 
working  on  SA3  and  changes  in  the  support  structure  we  put  in  place  over  the  last 
year.  The  predominance  of  support  used  to  be  provided  by  my  field  office.  Because  of 
personnel  changes  and  other  factors  we  switched  the  first  line  of  support  directly  to 
SIMA.  This  apparently  worked  to  our  benefit  It  pleased  the  users  and  freed  the  field 
office  to  pursue  their  original  mission  of  requirements  analysis  and  generation.  The  new 
staff  at  SIMA  and  the  new  systems  technicians  at  the  MSCk  never  knew  the  ‘old*  way  we 
did  business  and  so  they  responded  accordingly.  Those  that  did  remember  indicated  a 
preference  for  direct  contact  with  SIMA,  something  we  never  could  do  before,  because 
the  expertise  was  never  there;  it  was  only  with  my  field  office.  Other  issues  identified  in 
the  response  such  as  the  situation  room  and  the  user  manual  have  been  raised  before.  They 
are  isolated  problems  in  most  cases  sparked  by  a  bad  situation.  The  issue  of  testing  is 
addressed  in  the  next  question. 
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13.  Ho w  would  you  like  to  utilize  Mr.  Keeling's  office  In  tbe  future?  (Some 
example's^.) 

•  Continue  to  generate  new  requirements? 

*  Have  them  generate  all  requirements  Including  maintenance? 

-  Be  the  focal  point  for  all  communication  with  our  developers? 


Would  prefer  not  to  use  the  office.  Don't  want  a  "middle-man \  2  responses 

Test  and  evaluate  SA3  programs  in  process  (development).  Be  a  consolidation 
point  for  SA3  requirements. 


Be  focal  point  for  communication  with  developer's. 


Generate  new  requirements. 


Conduct  tests 


Conduct  tests  on  EUCPs. 


^responses 


ANALYSIS  AND  COMMENT*  The  response  reflect  the  changing  role  of  the  office 
from  a  front  line  support  office  to  a  requirements  analysts  and  design  office.  They 
arc  available  lo  assist  SIMA  in  testing  changes.  (  We  had  released  some  changes  that 
some  users  felt  were  not  tested  satisfactorily.)  They  see  the  field  office  as  a  redundant 
check  for  SIMA  to  make  sure  testing  is  complete.  The  responses  to  ‘not  use*  stem  from 
some  dealings  between  the  users  and  the  field  office  that  were  unpleasant,  and  that  was 
addressed  accordingly.  As  we  move  toward  a  DOD  standard  system  the  field  office  will 
take  on  a  greater  responsibility  as  the  liaison  lo  the  MSCs  and  generator  of  Army 
requirements  for  the  new  system.  They  have  knowledge  of  the  functional  and  technical 
aspects  of  the  current  system  that  is  critical  to  success. 
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UJ.  AMC  SYSTEMS  INTtQftATlOM  AMD  MANAOtMBKT  ACTIVITY 

us  spruce  rrecrr 
vr.  LoiMA.  mo.  ssim-mm 

MNVTO 
ArmmOM  Of 

AMXS1-Z 

MEMORANDUM  for  COMMANDER,  U.S.  ARMY  MATERIEL  COMMAND, 
ATTN:  AMCIR-A,  5001  EISENHOWER  AVENUE, 
ALEXANDRIA,  VA  22333-0001 

SUBJECT:  DODIG  Draft  Report,  Army  Procurement  and  Contract  Administration 
Practices  on  Computer  Software  Service  Contracts,  Project  4CF-5004  (AMC  No.  D9409) 

1 .  Reference  memorandum.  Department  of  the  Army,  Headquarters  U.S.  Army  Materiel 
Command,  14  Dec  94,  subject  as  above. 

2.  Comments  to  Adequacy  of  Technical  Support  Provided  bv  STMA^in2L-221 
♦Control  Bvnassed  on  System  Change  (nTft 

One  of  the  biggest  issues  was  that  there  was  no  System  Change  Request  (SCR)  associated 
with  the  conversion  of  SA3  from  S2K  to  M204.  Unfortunately,  the  project  manager  at 
the  time  retired  and  no  one  can  explain  the  circumstance  behind  this  situation.  However, 
this  was  a  legitimate  project.  The  auditors*  discussion  stated  that  “S  A3  needed  M204 
format  capabilities  for  the  system  to  be  used  at  the  AMC  Major  Subordinate  Commands." 
(p20).  Also,  a  survey  conducted  by  USASAC  during  Oct-Dec  1994  indicated  that  MSCs 
Sc  USASAC  use  M204  SA3  to  prepare  98.434  of  their  Letter  of  Offer  and  Agreement 
(LOA),  and  that  MSCs  use  M204  SA3  to  prepare  92.734  of  their  modifications  and 
amendments.  Therefore,  the  conversion  of  SA3  to  M2 04  was  a  needed  change. 

The  report  cited  that  SIMA  inappropriately  used  internal  projects.  Of  the  1 14  tasks 
reviewed,  the  report  cited  7  taskings  which  were  inappropriate.  Of  the  7,  three  were 
related  to  M204  conversion  of  SA3  which  should  be  associated  with  a  SCR.  From  the 
titles  of  the  remaining  projects,  they  appeared  to  be  projects  that  were  not  associated  with 
any  system  modifications.  For  example,  EA-USS-SA3  and  HQ  AMC  HSl  S  A3  were 
projects  associated  with  supporting  the  Executive  Agent  and  the  HQ  AMC  Functional 
System  Integration  group  for  data  calls,  inquiries,  meetings,  etc.  SA3-S  AMAP  was  a 
project  in  support  of  a  MICOM  developed  system 

♦Control  Bypassed  on  Contractor  Work  Order.  (d231 

Since  July  1993,  a  new  SIMA  client  representative  was  assigned  to  the  Security 
Assistance  Automation  project,  since  then,  work  orders  were  properly  processed  against 
the  S  A3  Technical  Support  Task  Order  specifying  legitimate  workload  and  requirements. 
Only  a  minimal  amount  of  work  orders  were  processed  for  customer  support  type 
requirements  which  docs  not  associate  with  any  SCR,  366a' s  or  User  Test. 
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♦SIMA  Failure  tn  Fnfnrce  Iocistiny  Guidance  (r,74\ 

The  report  indicated  that  SIMA  did  not  prepare  design  documentation.  Detailed  design 
documentation  'was  prepared  for  all  software  with  the  exception  of  Case  Print 

*lnadefluaig  Customer  SiioaorL  £o2fl 

Since  early  1994,  SlMA/USASAC  redefined  responsibilities.  SIMA  gained  the 
responsibility  to  directly  support  the  customer.  The  survey  conducted  by  US  AS  AC 
during  Oct-Dcc  1994,  the  following  question  was  asked  of  the  customer: 

How  would  you  rate  the  support  (i.e.,  training,  customer  response,  problem  solving) 
you  have  received  over  the  years  from  the  following  activities?  (l=poor,  2*=5atisfactory, 
3=good,  4=cxccllcnt) 

SIMA  received  a  rating  of  3 .9. 

♦Canabilities/dcficiencics  nfthe  Contractor-developed  versus  SIMA-develoned  M2Q4 
casenrint  process.  (v25  and  appendix  Q 

The  report  failed  to  mention  that  the  case  print  process  developed  by  the  SIMA 
programmer  did  not  work  with  Mu  del-37  laser  printer  which  was  a  critical  factor. 

Also,  the  test  coses  that  were  run  at  US  AS  AC-New  Cumberland  were  not  representative 
of  the  cases  that  are  run  at  the  MSCf.  Therefore,  one  could  not  conclude  that  the  case 
print  process  developed  by  the  SIMA  programmer  had  only  minor  deficiencies. 

3.  The  Findings  and  Recommendations  are  as  follows: 

Finding:  SIMA  did  not  follow  established  procedures  or  provide  adequate  technical 
support  during  SA3  development  Specifically,  SIMA  did  not  always  obtain  system 
change  requests  or  prepare  planning  and  design  documentation  as  required  before 
modifying  and  enhancing  SA3.  In  addition,  SIMA  failed  to  perform  key  functions  during 
SA3  development  because  SIMA  management,  in  some  instances,  allowed  the  Security 
Assistance  Functional  Support  Office,  US  ASAC,  to  perform  duties  that  overlapped  and 
interfered  with  SIMA’s  responsibility  as  a  central  design  activity. 

Recommendation  3a:  Develop  additional  internal  control  objectives  and  techniques  on 
the  documentation  requirement  for  the  Security  Assistance  Automation,  Army 
information  system. 
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ACTION  TAKEN:  Concur.  SIMA  will  develop  internal  control  objectives  and 
techniques  on  the  documentaiton  requirement  Tor  the  Security  Assistance  Automation, 
Army  information  system.  A  target  date  of  31  March  1995  has  been  established. 

Recommendation  3b:  Establish  procedures  on  the  use  of  internal  projects  and  define  and 
limit  the  use  of  internal  projects  to  administrative  support  functions  for  all  information 
systems. 

ACTION  TAKEN:  Concur  with  the  establishment  of  procedures  on  the  use  of  internal 
projects.  Nonconcur  on  the  limitation  of  these  projects  to  administrative  support 
functions  only. 

The  terminology  “internal  projects”  is  misleading.  SIMA's  Resource  Management 
System  (RMS)  consists  of  individual  work  projects.  Not  all  projects  are  associated  with 
System  Change  Requests  (SCRs).  The  projects  that  have  no  SCRs  are  not  necessarily 
administrative  support  functions,  e.g.  legitimate  design  and  development  workload  in 
support  of  the  Joint  Logistics  Systems  Center.  SIMA  also  responds  to  many  data  calls, 
customer  inquiries,  support  work  for  system  software,  release  management  and  etc.  None 
of  these  have  SCRs.  We  cannot  restrict  projects  to  only  administrative  support  functions. 
A  target  date  of  31  March  1995  for  the  establishment  of  procedures  has  been  established. 

Recommendation  3  c:  Develop  and  issue  policy  prohibiting  the  acceptance  of  work  to 
enhance  or  modify  the  Security  Assistance  Atuomation,  Army  information  system  unless 
a  system  change  request  accompanies  the  work  request. 

ACTION  TAKEN:  Concur.  SIMA  will  develop  and  issue  a  policy  statement  prohibiting 
the  acceptance  of  work  to  enhance  or  modify  the  Security  Assistance  Automation.  Army 
information  system  unless  a  system  change  request  accompanies  the  work  request.  A 
target  date  of  28  February  1995  has  been  established. 

Recommendation  3d:  Develop  and  issue  policy  discontinuing  the  practice  of  submitting 
work  orders  for  Security  Assistance  Automation,  Army  information  system  to  the 
contractor  without  required  supporting  documentation. 

ACTION  TAKEN:  Concur.  SIMA  will  develop  and  issue  a  policy  statement  that  work 
orders  submitted  requesting  contract  support  for  systems  changes  to  Security  Assistance 
Automation,  Army  information  system  will  require  supporting  documentation.  A  target 
date  of  28  February  1995  has  been  established. 

Recommendation  3c:  Create  a  formal  plan,  including  design  and  testing  requirements, 
before  performing  any  additional  Security  Assistance  Automation,  Army  information 
system  development,  maintenance,  or  modification. 

ACTION  TAKEN:  Concur.  SIMA  will  insure  that  a  project  plan  is  developed  for  each 
software  release.  A  target  date  of  1  February  1995  has  been  established. 
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ns  should  arise,  please  contact  Danwn  Thompson  at  DSN  555-4219. 

wd  all  ends  AsL'J - 

tJ  loimnnelledge 

V  Director,  SIMA-Wcst 


Director,  SIMA-Wcst 
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